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A  PRELIMINARY  MANPRINT  EVALUATION  OF  THE 
ALL  SOURCE  ANALYSIS  SYSTEM  (ASA S) 


SYSTEM  DESCRIPTION 

/  •  i'  C 

/  S  \ 

The  Ail  Source  Analysis  System  (ASAS)  is  being  designed  to  provide 
commanders  with  a  transportable,  semiautomatic,  tactical  intelligence  system 
capable  of  producing  an  integrated  picture  of  the  battlefield.  -'In  addition  to 
system  supervision,  functions  to  be  provided  include  intelligence  development, 
situation  development,  target  development,  collection  management,  electronic 
warfare/electronic  combat  support,  and  operations  security  support.  However, 
the  system  evaluated  during  the  field  trials  at  Fort  Hood,  TX,  had  a  greatly 
reduced  functional  capability.  In  fact,  only  one  of  the  modules,  the  Forward 
Sensor  Interface  and  Control  (FSIC),  was  configured  as  a  part  of  the  objective 
ASAS  system.  The  items  employed  during  the  trials  were  operated  as  part  of  a 
Division  communication  net.  The  net  included  the  forward  sensor  communicating 
to  the  Military  Intelligence  (MI)  Battalion  which  communicated  to  the  Division 
Tactical  Operations  Center  (DTOC)  Support  Element.  Under  this  net  concept, 
intelligence  information  from  the  forward  sensors  and  jammers  is  relayed  to  the 
MI  Battalion  and  DTOC  support  elements,  and  requirements  or  tasking  for 
additional  information  and  data  flows  back  through  the  system  to  the  forward 
elements.  Further  description  of  the  actual  configuration  employed  during  the 
field  trials  is  given  below. 

Forward  Sensors 


Since  the  exercises  controlled  by  the  Operational  Test  and  Evaluation 
Agency  (OTEA)  followed  a  given  scenario,  no  actual  Forward  Sensors  were 
employed.  Messages  representing  several  surrounding  sensors  were  generated  by 
the  TACSIM  computer,  printed  on  input  forms  in  the  proper  format  and  then  taken 
to  the  Sensor  Input  Station.  They  were  inserted  into  the  system  at  the  proper 
scenario  time  via  the  AN/PSC-2  digital  communications  terminal  (DCT) .  Simulated 
TRAILBLAZER  inputs  were  entered  directly  into  the  Forward  FSIC. 

Forward  FSIC 


The  forward  FSIC  served  as  the  collection  point  for  all  messages  from  the 
forward  sensors  and  jammers.  Most  such  messages  were  automatically  relayed  to 
the  rear  by  means  of  secure  radio  communications.  The  forward  FSIC  also  served 
as  the  relay  point  for  tasking  and  other  messages  from  the  rear  to  the  forward 
sensors.  Employment  of  the  Fully  Automatic  Routing  of  Messages  (FARM)  software 
permitted  messages  to  be  automatically  routed  to  one  or  several  addresses. 

Also,  the  software  could  be  employed  to  identify  messages  that  had  combat 
information  of  immediate  use  to  Brigade  command  staff.  These  messages  were 
routed  to  the  FSIC  operator's  queue,  sanitized  to  an  appropriate  level,  and 
then  transmitted  by  voice  radio  to  Brigade  command  staffs.  Messages  which  were 
improperly  formatted  or  improperly  addressed  were  also  automatically  routed  to 
an  operator's  queue  for  correction  and  retransmission.  The  FSIC  is  mounted  in 
an  S250  shelter  transported  on  a  Commercial  Utility  Cargo  Vehicle  (CUCV).  A 
towed  12.5  kw  generator  provides  power.  The  primary  elements  of  the  FSIC 
hardware  include  an  RM  11/84  computer,  a  GRIDSE-T  model  1179  operator  terminal, 
a  dual  disk  drive,  a  tape  cartridge  unit,  two  printers,  five  radios  with 
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associated  security  devices,  and  a  DCT.  The  DCT  permits  the  FSIC  operator  to 
send  or  receive  preformatted  or  free  text  messages  to  or  from  the  frontline 
sensors. 

Rear  Combat  Electronic  Warfare  and  Intelligence  (CEWI)  FSIC 

The  rear  FSIC  is  identical  in  construction  to  the  forward  FSIC  and  enables 
the  same  functions.  But  its  operator  typically  performs  many  fewer  functions 
than  the  operator  in  the  forward  FSIC.  Most  messages  requiring  operator 
intervention  are  handled  in  the  forward  FSIC.  All  messages  from  the  forward 
FSIC  (except  those  sent  by  voice  directly  to  the  Brigade  liaison  officer)  are 
received  by  the  rear  FSIC  and  passed  to  the  ASAS  Interface  Module  (AIM) . 
Similarly,  all  tasking  and  technical  data  messages  from  rear  operators  are 
passed  through  the  rear  FSIC  to  the  forward  FSIC  for  transmission  to  the 
sensors  and  jammers. 

CEWI  Tactical  Control  and  Analysis  Element  (TCAE)  AIM(6) 

The  AIM  module  consists  of  a  VAX  750R  computer  and  associated  equipment 
which  is  mounted  in  an  S250  shelter  and  transported  by  a  CUCV.  The  computer 
collects  and  stores  messages  and  data  received  from  the  forward  sensors  and 
jammers.  This  data  can  be  accessed  by  the  Intelligence  analysts.  The  (6)  in 
the  AIM(6)  refers  to  the  six  operator  stations  which  are  housed  in  three 
Modularized  Equipment  Container  Assemblies  (MECAs).  Five  of  the  work  stations 
contain  MICROFIX  terminals  and  the  sixth,  for  the  system  supervisor,  contains  a 
TER- 100  terminal.  Two  of  the  analyst  work  stations  were  for  communications 
intelligence  analysis,  two  for  electronic  intelligence  analysis,  and  one  for 
assets  management.  The  stations  were  operated  by  members  of  the  MI  Battalion 
technical  control  and  analysis  element  (TCAE).  Although  the  MECAs  were  Intended 
to  be  transported  on  a  CUCV,  during  the  trials  they  were  mounted  in  a  five-ton 
expandable  van.  A  third  CUCV  was  employed  to  haul  ancillary  equipment  during 
movement,  and  two  towed  12.5  kw  generators  were  employed  to  provide  power  to  the 
system. 

DTOC  AIM( 6 ) 

The  hardware  at  the  DTOC  was  identical  to  that  operated  by  the  TCAE. 
However,  the  functions  performed  by  the  operators  were  different.  In  addition 
to  a  system  supervisor  position,  there  were  two  positions  for  situation 
analysts,  and  one  each  for  collection  management,  target  assessment  and  All 
Source  Analysis.  The  equipment  at  the  DTOC  was  operated  by  personnel  assigned 
to  the  Division  G-2. 

It  should  be  noted  that  neither  the  work  stations  nor  the  AIM  module 
itself  are  part  of  the  objective  (planned)  ASAS  hardware.  Smaller,  much 
lighter  Portable  Analyst  Work  Stations  (PAWS)  will  replace  the  MICROFIX 
stations,  and  a  new  AIM  module,  designed  to  interface  between  the  FSIC  and  the 
work  stations,  will  replace  the  AIM  employed  during  the  field  trials.  Also, 
the  software  for  the  field  trials  was  version  RA  which  allowed  only  limited 
system  functioning.  The  version  of  the  RA  release  employed  included  changes  to 
address  tactical  requirements  of  the  2nd  Armored  Division  as  identified  during 
earlier  testing.  Software  will  be  improved  throughout  the  development  of  the 
system  and  will  include  multiple  releases. 


CONDUCT  OF  THE  TRIALS 


The  Initial  operator  training  was  conducted  at  the  contractor's  facility 
during  August  and  September  1986*  Two  groups  of  personnel  were  trained  in 
separate  two-week  training  sessions.  Training  was  followed  by  a  system 
readiness  verification  test  (SRVT),  also  conducted  at  the  contractor's  facility. 
Following  the  SRVT,  the  system  hardware  was  transported  to  Fort  Hood,  Texas,  and 
reassembled  in  a  garrison  environment.  Data  collector  training  was  conducted  in 
early  November  with  practice  in  mid-November  during  a  2d  Armored  Division 
exercise.  Second  Armored  Division  personnel,  who  rotated  as  players  and  data 
collectors  later  during  the  field  trials,  received  training  on  data  collection 
prior  to  assignment  as  data  collectors.  A  pilot  test  was  conducted  shortly 
after  the  middle  of  November  to  determine  whether  operator  training,  data 
collection  procedures,  data  management  procedures,  and  system  availability  were 
adequate  to  meet  the  objectives  of  the  later  phases. 

The  second  phase  of  the  trials  consisted  of  two  command  post  exercises 
(CPX)  conducted  during  December  1986.  The  CPXs  consisted  of  approximately  90 
hours  of  system  operation  with  the  system  being  driven  by  TACSIM-derived 
outputs.  The  test  directorate  controlled  the  trials  during  both  of  the  CPXs. 

A  third  phase,  a  field  training  exercise  (FTX),  was  controlled  by  III  Corps/2d 
Armored  Division,  and  consisted  of  approximately  77  hours  of  system  deployment 
and  operation  in  maneuver  areas  on  Fort  Hood.  The  FTX  was  also  conducted  in 
December  1986. 


TEST  ISSUE 


The  test  issue  addressed  in  this  report  was  issue  six,  MANPRINT:  what  are 
the  MANPRINT  Implications  of  the  AIM(6)/FSIC?  The  test  director  was  aware  that 
the  domains  of  manpower,  personnel  and  training  would  be  addressed  only 
superficially  in  the  data.  There  were  several  reasons  for  this.  First,  most 
of  the  hardware  system  components  involved  were  substitutes.  Second,  the 
software  available  was  a  very  preliminary  version.  Third,  the  overall  system 
configuration  was  very  limited.  Under  these  circumstances,  it  was  felt  that 
neither  the  operator  personnel,  their  Immediate  supervisors,  nor  the  test 
directorate  personnel  could  realistically  evaluate  manpower  requirements, 
essential  personnel  characteristics,  nor  the  exact  nature  and  extent  of  training 
required.  On  the  other  hand,  it  was  felt  that  the  domains  of  human  factors 
engineering,  health  hazards  assessment  and  system  safety  assessment  should  be 
covered  to  the  extent  possible  in  order  to  provide  data  for  consideration  in 
hardware/sof tware  design  of  the  eventual  system  components. 

It  was  further  understood  that  priority  should  be  given  to  the  evaluation 
of  the  FSIC  component  of  the  ASAS.  Any  major  deficiencies  found  in  the  FSIC 
would  have  to  be  considered  for  modification.  Data  on  deficiencies  and 
strengths  of  the  stand-in  modules  would  be  provided  to  the  designers  of  the 
future  versions  of  those  modules. 


DETAILS  OF  THE  TEST 


Data  Collection  and  Analysis  Plan. 


Early  plans  called  for  data  pertinent  to  the  MANPRINT  domains  to  be 
obtained  from  questionnaires,  interviews,  rating  scales,  test  personnel  and 
data  collector  comments,  relevant  performance  data  from  the  OTEA  database, 
physical  measurements  of  selected  human  engineering  characteristics  and  direct 
observations  by  the  staff  member.  Copies  of  all  data  collection  forms  and 
requirements  are  shown  in  Appendixes  A  through  J.  All  data  collection  forms 
were  submitted  to  the  test  directorate  for  review  and  comment,  and  revised  as 
requested.  Items  selected  for  the  end-of-test  structured  interviews  were  those 
thought  to  be  difficult  to  cover  with  a  group-administered  questionnaire. 
However,  it  was  anticipated  that  the  structured  interviews  would  be  revised 
based  on  both  the  direct  observations  during  the  trials  and  the  operators' 
responses  to  the  questionnaires.  Although  a  general  methodology  and  data 
collection  plan  had  been  coordinated  with  the  test  directorate,  the  actual  data 
collection  forms  were  not  devised  until  the  staff  member  had  observed  operator 
training  and  had  the  opportunity  to  review  both  the  task  analyses  and  the 
written  training  materials  provided  by  the  contractor.  In  addition,  several 
items  for  inclusion  in  the  questionnaires  were  submitted  by  test  directorate 
personnel . 


It  was  assumed  from  the  beginning  that  descriptive  rather  than  inferential 
statistics  would  be  the  primary  mode  of  analysis.  There  were  two  primary 
reasons  for  this.  First,  there  was  no  real  predecessor  system  with  which  to 
compare  the  system  under  trial.  Both  the  Battlefield  Exploitation  and  Target 
Acquisition  (BETA)  system  and  the  Technical  Control  and  Analysis  Center- 
Division  (TCAC-D)  provided  limited  support  to  signals  intelligence  functions. 
However,  neither  of  these  systems  provides  the  full  array  of  capabilities 
identified  as  ASAS  requirements  in  the  Required  Operational  Capability  (ROC) 
document.  Second,  the  number  of  individuals  who  performed  any  one  of  the 
specific  operator  or  analyst  jobs  during  the  trials  was  very  small.  For 
example,  while  there  were  20  MICROFIX  station  operators  (two  shifts  of  five 
operators  each  at  both  the  TCAE  and  the  DTOC),  they  performed  seven  different 
jobs.  Four  FSIC  operators  performed  jobs  in  the  forward  FSIC,  and  another  four 
worked  in  the  rear  FSIC.  The  workload  of  the  forward  FSIC  operator  was 
acknowledged  to  be  both  heavier  and  to  include  additional  tasks  not  performed 
by  operators  in  the  rear  FSIC.  Therefore,  considering  the  small  number  of 
individuals  who  performed  any  one  job,  statistical  comparisons  of  the  effective¬ 
ness  of  the  system  among  the  job  positions  or  even  between  the  TCAE  and  DTOC 
would  have  been  of  low  value . 


While  these  more  typical  comparisons  were  ruled  out,  some  specific  plans 
for  the  treatment  of  the  data  were  made.  These  were: 

One,  an  examination  of  the  relationships  among  self-reported  demographic 
data,  responses  to  specific  questionnaire  items,  and  ratings  by  supervisors. 
It  was  assumed,  for  example,  that  any  training  or  experience  with  computer 
systems  (Including  either  BETA  or  TCAC-D)  should  be  related  in  some  manner  to 
supervisors'  judgments  of  job  performance  variables,  relevant  knowledge,  and 
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job/system  attitudes.  Similarly,  it  was  assumed  that  experience  as  measured  by 
time  in  MOS  should  show  some  relationship  to  the  supervisors'  ratings.  However, 
no  direction  for  any  of  the  relationships  was  assumed.  It  was  thought  possible, 
for  example,  that  the  operators  with  more  computer  experience,  while  performing 
more  adequately,  might  see  more  shortcomings  in  the  system  and  give  it  lower 
ratings.  Similarly,  speculative  reasons  why  job  knowledge  and  experience  could 
affect  job  performance  and  attitudes  either  positively  or  negatively  could 
readily  be  postulated. 

Two,  tabulation  of  all  responses  to  the  multiple-choice  items  of  each  of 
the  questionnaires  was  planned.  Separate  tabulations  of  responses  for  personnel 
in  the  DTOC  and  TCAE  were  planned,  as  were  separate  tabulations  for  operators  in 
the  forward  and  rear  FSICs.  However,  it  was  felt  that  responses  for  all  eight 
FSIC  operators  could  be  combined  for  most  items  as  could  the  responses  for  all 
of  the  AIM(6)  analysts.  This  is  because  it  seemed  reasonable  that  personnel 
operating  Identical  equipments  would  have  similar  man/machlne  interface,  safety 
and  health  hazard  problems.  Also,  it  was  known  from  observations  during 
training  and  discussions  with  operators,  that  they  frequently  consulted  each 
other  concerning  problems  with  particular  functions,  and  worked  together  to 
solve  problems.  By  combining  the  responses  the  sample  sizes  could  be  increased, 
and  presumably,  make  the  data  more  meaningful. 

Three,  an  analysis  of  both  the  written  comments  from  the  questionnaires 
and  those  obtained  during  the  interviews  was  made.  It  was  expected  that 
written  comments  on  the  questionnaires  would  clarify  the  meanings  of  the 
responses  made  by  the  individual  respondents  to  the  multiple-choice  items.  It 
was  also  expected  that  the  comments  might  reveal  potential  or  existing  strengths 
or  weaknesses  either  not  covered  at  all  or  not  covered  adequately  by  the  written 
items.  The  same  was  anticipated  for  much  of  the  commentary  obtained  during  the 
interviews.  In  other  words,  the  comments  were  to  be  used  to  ’’fill  in  the 
blanks"  and  help  ensure  that  all  data  pertinent  to  the  system  that  could  be 
obtained  from  the  trials  was  obtained.  It  was  also  assumed  that  most  of  the 
data  relevant  to  the  MANPRINT  domains  of  manpower,  personnel  and  training  would 
come  from  the  commentaries. 

Limitations 


Significant  Factors.  Several  factors  had  a  significant  influence  on  the 
conduct  of  the  MANPRINT  evaluation.  These  were  beyond  the  control  of  either  the 
MANPRINT  evaluator  or  the  test  directorate,  and  except  for  the  ever  present 
possibility  of  losing  important  data,  were  not  anticipated.  Some  constraints  on 
the  planned  treatment  of  the  data  did  not  become  evident  until  the  trials  were 
nearly  over.  Nevertheless,  they  had  a  negative  Impact  on  the  evaluation.  The 
exact  extent  of  the  impact  cannot  be  accurately  determined,  but  there  is  little 
doubt  that  it  was  serious. 

Lack  of  observation  of  ln-FSIC  training.  Only  two  FSIC  modules,  one  AIM 
module  and  two  sets  of  analysts'  stations  were  available  for  training.  Because 
priority  was  given  to  the  FSIC,  the  MANPRINT  evaluator  attended  FSIC  operator 
training  classes  as  an  observer.  Although  allowed  to  participate  fully  in  the 
classroom,  only  the  military  personnel  who  were  to  serve  as  operators  received 
hands-on  training  inside  the  FSIC  shelter.  In  fact,  it  was  virtually  impossible 
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even  to  observe  the  hands-on  training.  Since  four  operators  were  to  be  trained, 
two  operators  and  an  instructor  were  inside  the  FSIC  shelter  at  all  times  when 
the  equipment  was  functioning.  These  three  crowded  the  very  limited  space 
available  in  the  FSIC  shelter,  making  it  impossible  to  observe  the  actual 
operation.  Furthermore,  the  high  noise  level  (measured  later  at  76  dBA)  made 
it  nearly  impossible  to  hear  the  verbal  exchanges  while  standing  outside  at  the 
entry.  As  a  result,  the  MANFRINT  evaluator  was  only  able  to  obtain  information 
on  operator  problems  via  discussions  with  the  operators  during  breaks,  lunch 
periods,  and  during  periods  of  system  malfunction.  This  lack  of  first-hand 
experience,  even  direct  observational  experience,  made  it  difficult  to  ferret 
out  existing  or  potential  man/machlne  interface  problems  or  system  design 
deficiencies. 

Lack  of  observation  during  the  field  trials.  After  assignment  to  the  ASAS 
evaluation,  there  was  insufficient  time  for  the MANPRINT  evaluator  to  obtain  a 
clearance  for  access  to  Sensitive  Compartmented  Information  (SCI).  Some 
messages  passing  through  the  system  during  the  field  trials  were  at  the  SCI 
level.  For  this  reason,  it  was  determined  by  security  personnel  that  the 
MANPRINT  evaluator  could  not  observe  operations  while  the  system  was  energized. 
The  result  was  that  the  only  functions  the  evaluator  was  able  to  observe 
directly  were  displacement  and  emplacement.  The  lack  of  the  special  intel¬ 
ligence  clearance  also  had  a  detrimental  effect  on  the  interviews.  There  were 
some  aspects  of  the  system  which  the  operators  were  hesitant  to  discuss  with 
the  interviewer  for  fear  of  divulging  information  the  interviewer  was  not 
authorized  to  obtain. 

Unavailability  of  requested  system/operator  performance  data.  The  data 
requested  from  the  OTEA  database  covered  several  areas,  but  a  major  purpose  for 
the  specific  data  requests  was  to  evaluate  the  effects  of  various  factors  on 
operator  workload.  Unfortunately,  very  little  of  the  data  could  be  made 
available.  Part  of  the  problem  was  due  to  the  late  date  at  which  the  MANPRINT 
evaluator  joined  the  test  team.  The  level  of  effort  programmed  for  the  database 
contractor  had  already  been  determined  before  the  MANPRINT  requirements  could  be 
formulated,  and  the  resources  to  meet  the  requirements  were  simply  not  avail¬ 
able.  Most  of  the  data  requested  would  have  required  the  correlation  of  data 
from  multiple  sources.  For  example,  several  items  (items  C,  D,  E,  and  F  in 
Appendix  J)  were  designed  to  assess  backlog  in  message  transmissions  as 
indicators  of  operator  workload  in  the  FSIC.  For  these  data  to  be  meaningful, 
it  would  have  been  necessary  to  relate  these  data  to  message  input  rate  at  the 
times  in  question.  It  would  also  have  been  necessary  to  determine  whether  the 
backlogs  developed  were  due  to  failure  of  transmission  lines  or  software 
failures  which  prevented  the  messages  from  moving  through  the  system.  Providing 
these  data  would  have  required  the  development  of  additional  software  programs. 
The  database  contractor  determined  that  time  and  resource  constraints  precluded 
the  development  of  the  necessary  software  to  provide  the  data  in  the  form 
needed.  The  time  required  for  the  MANPRINT  evaluator  to  manually  extract  the 
data  was  also  beyond  that  available.  It  was  felt  that  without  being  able  to 
take  into  account  those  factors  which  influenced  the  basic  data  requested,  that 
these  data  could  easily  be  misinterpreted.  Therefore,  the  decision  was  made  to 
present  no  data  from  the  system  performance  database. 
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Data  voids  or  near  voids.  Supervisors  were  asked  to  rate  their  personnel 
on  a  number  of  performance,  attitude,  and  knowledge  items.  The  intent  was  to 
examine  the  relationships  between  these  ratings  and  selected  demographic 
variables  and  questionnaire  responses.  However,  many  of  the  supervisors  did 
not  complete  the  rating  forms,  saying  that  they  were  unable  to  realistically 
rate  the  characteristics  described.  Where  ratings  were  received,  the  raters 
tended  to  use  only  the  top  two  categories,  providing  very  little  discrimination 
between  incumbents.  Also,  in  one  instance  where  two  sets  of  ratings  were 
received  for  the  same  personnel,  the  supervisors  showed  low  agreement.  The 
combination  of  lack  of  discrimination,  low  agreement,  and  missing  data  rendered 
the  ratings  of  insufficient  value  for  presentation. 

Demographic  Relationships.  The  brief  biographical  form  included  in  the 
questionnaires  requested  the  operators  to  provide  information  on  experience  and 
training  on  automated  systems.  It  had  been  thought  that  differences  in  this 
area  might  influence  questionnaire  responses  and  also  be  related  to  supervisors' 
opinions.  However,  none  of  the  operators  reported  any  special  training  on 
computer  systems;  none  had  any  experience  with  TCAC-D,  and  only  three  (two  AIM 
and  one  FSIC)  operators  owned  personal  computers.  These  numbers  were  considered 
to  be  too  small  to  warrant  any  comparisons  based  on  differences  in  experience  or 
training  with  automated  systems.  Time  in  MOS  as  a  measure  of  experience  was 
also  included.  However,  during  the  interviews,  the  operators  indicated  that 
neither  their  MOS  training  nor  their  MOS  experience  was  very  relevant  to  their 
jobs  on  the  ASAS  system.  This  point  was  especially  emphasized  by  FSIC  operators 
who  stated  that  a  communications  specialist  rather  than  an  intelligence  analyst 
would  be  a  more  appropriate  choice  for  the  FSIC  operator.  Based  on  these 
findings,  any  plans  to  look  for  relationships  of  the  demographic  data  to  other 
data  were  abandoned. 

Data  Collection  Procedures 


The  handout  describing  the  MANPRINT  domains  (Appendix  H)  was  given  to  the 
data  collectors,  test  directorate  personnel,  and  personnel  supervisors  during 
the  first  week  of  the  trials.  Operator  questionnaires  (Appendixes  A  and  B) 
were  administered  in  groups  near  the  completion  of  the  second  CPX.  Operators 
not  available  at  scheduled  times  completed  the  questionnaires  during  time 
available  the  following  week. 

Supervisor  rating  forms  (Appendixes  F  and  G)  for  operators  were  given  to 
the  supervisors  at  the  beginning  of  the  last  week  of  the  trials  and  collected 
by  the  test  directorate  at  the  end  of  the  week.  The  same  was  true  of  the 
Personnel  Supervisor/Observer  questionnaires.  "Personnel  supervisors"  were 
those  in  the  command  structure  who  did  not  directly  supervise  the  operators 
during  job  performance.  "Observers"  receiving  the  questionnaire  were  selected 
by  the  test  directorate.  They  were  not  directly  involved  in  the  system 
operations  but  were  frequently  observing  on  site  and  were  highly  familiar  with 
the  system.  Personnel  not  returning  the  forms  were  contacted  the  following 
week  in  an  attempt  to  collect  as  much  of  the  data  as  possible. 
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Operator  interviews  (Appendixes  C  and  D)  were  conducted  with  groups  of  two 
to  four  personnel  during  the  last  week  of  the  trials  with  the  exception  of  one 
AIM  and  two  FSIC  operators  who  were  interviewed  individually  after  the  trials 
were  completed.  Due  to  reasons  beyond  the  control  of  the  test  directorate,  two 
of  the  AIM  operators  were  not  available  for  the  final  interviews.  All  eight  of 
the  FSIC  operators  were  interviewed. 

The  physical  measurements  (Appendix  I)  obtained  for  the  FSIC  were  completed 
with  the  aid  of  an  operator  in  the  CEWI  FSIC  the  week  following  the  completion 
of  the  trials. 

The  MANPRINT  evaluator  attended  meetings  of  the  Data  Analysis  Group  (DAG) 
which  were  held  almost  daily  during  the  first  few  days  of  the  trials  and  weekly 
thereafter.  The  purpose  of  the  meetings  was  to  verify  and  validate  the  data 
management  procedures,  to  review  and  verify  data  base  entries,  and  to  inves¬ 
tigate  any  problems  that  surfaced  during  the  conduct  of  the  test. 


RESULTS 

The  primary  purpose  of  any  evaluation  of  a  system  under  development  is  to 
provide  decision  makers  with  objective  data  on  which  to  which  to  base  decisions 
concerning  the  system's  future.  Assuming  that  the  development  is  continued, 
the  data  are  of  primary  use  to  the  system's  developers.  To  be  most  useful  to 
developers,  the  data  should  not  only  point  out  actual  or  potential  deficiencies 
that  suggest  a  need  for  system  modifications,  but  should  also  indicate  strengths 
where  modifications  should  not  be  made.  With  this  in  mind,  the  data  were 
examined  to  learn  what  aspects  of  the  system  the  users  considered  positive  and 
which  they  considered  negative. 

The  remainder  of  this  results  section  is  divided  into  four  subsections. 

The  first  deals  with  the  results  derived  from  the  operator  questionnaires,  the 
second  with  findings  based  on  the  interviews,  the  third  with  the  responses  to 
the  supervisor/observer  questionnaire,  and  the  fourth  reports  findings  from  the 
measurements  taken  in  the  FSIC  shelter. 

The  first  part  of  each  subsection  consists  of  an  explanation  of  how  the 
data  were  handled.  The  second  part  presents  results  in  a  bulleted  format.  The 
third  part  is  a  discussion  which  attempts  to  synthesize  the  results  reported 
into  a  meaningful  whole. 


Operator  Questionnaire  Findings 


Data  Handling.  Tabulations  of  the  responses  to  the  questionnaires  are 
provided  in  the  appendixes.  Responses  to  the  FSIC  operator  questionnaire  are 
shown  separately  for  the  forward  and  rear  FSIC  operators.  Response  tabulations 
to  the  AIM  operator  questionnaire  are  shown  separately  for  DTOC  and  TCAE 
analysts,  with  a  total  which  includes  the  system  supervisors.  Although  there 
were  items  that  were  not  applicable  to  AIM  supervisors,  in  some  cases  the 
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supervisors  did  respond  to  these  Items.  Their  responses  are  Included  in  the 
totals. 

Positive  and  Negative  Responses.  It  was  assumed  that  unanimity  of  response 
would  be  rare.  Therefore,  an  arbitrary  decision  had  to  be  made  on  what 
constituted  positive  and  negative  responses.  For  purposes  of  summarizing  the 
major  findings,  responses  were  considered  to  be  positive  if  75  percent  or  more 
of  the  potential  respondents  reported  that  a  function,  purpose,  task  or  hardware 
item  was  at  least  "good,"  "easy,"  or  "adequate."  Responses  were  considered  to 
be  negative  if  25  percent  or  more  gave  ratings  of  “poor,"  "inadequate," 
"difficult,"  or  worse.  The  rationale  for  these  numbers  is  based  on  the  notion 
that  any  aspect  or  characteristic  of  the  system  which  is  rated  acceptably  by  at 
least  75  percent  of  the  users  is  probably  one  which  the  developer  would  want  to 
maintain,  and  therefore,  should  be  alerted  to  ensure  that  no  degradation  occurs 
in  subsequent  development.  On  the  other  hand,  if  some  aspect  or  characteristic 
is  seen  as  being  essentially  unacceptable  by  25  percent  or  more  of  the  users,  a 
further  look  at  the  problem  seems  warranted.  It  should  be  noted  that  by  these 
criteria,  responses  to  an  item  could  fall  into  both  the  positive  and  negative 
categories.  However,  it  was  assumed  that  this  would  be  most  unlikely  because  of 
the  available  "marginal"  and  “in  between"  response  alternatives. 

Yes  and  No  Items.  Not  all  of  the  items  in  the  questionnaires  required 
responses  like  those  described  in  the  preceding  paragraph.  In  some  cases, 
simple  “yes”  or  "no"  answers  were  required.  Some  of  these  items  were  included 
mainly  to  clarify  reasons  for  choices  to  previous  items.  Responses  to  these 
will  be  included  in  the  discussion  of  the  primary  finding  when  appropriate. 
However,  in  some  cases  the  yes/no  items  were  intended  to  "stand  alone"  because 
only  dichotomous  responses  seemed  reasonable.  These  items  will  be  Included  in 
the  summary  if.  In  the  judgment  of  the  MANPRINT  evaluator,  they  could  have  a 
significant  impact  on  system  design,  personnel  selection  or  training.  Still 
other  items  required  the  respondents  to  estimate  frequencies  of  occurrence. 
Responses  to  these  items  will  be  included  in  the  summary  if  it  is  judged  that 
they  might  impact  in  the  MANPRINT  domains. 

FSIC  operator  findings.  These  are  shown  below  in  bullet  format. 

°  Over  half  of  the  FSIC  operators  Indicated  that  they  should  have 
been  given  instruction  on  the  operation  of  the  KG84A  (digital 
encryptor),  the  TD1289  (digital  multiplexer),  and  the  C6533  (radio 
amplifier  and  switcher)  instrument  components  of  their  work 
stations.  More  of  the  forward  FSIC  operators  Indicated  this  need 
for  instruction  than  did  the  CEWI  FSIC  operators.  The  operators 
were  not  asked  about  previous  experience  with  these  elements,  so 
any  effects  from  experience  or  lack  thereof  cannot  be  ascertained. 

°  Six  of  the  eight  FSIC  operators  indicated  that  they  lost  "sync" 
three  or  more  times  on  the  average  during  a  12-hour  shift.  Three 
of  the  operators  indicated  that  sync  was  lost  on  the  average  of 
five  to  six  times.  All  eight  operators  indicated  that  it  was  easy 
to  reestablish  sync,  although  three  indicated  that  it  was  time 
consuming.  Written  comments  suggested  that  it  was  typically 
reestablished  quickly,  but  could  run  into  hours. 


°  Seven  of  Che  eight  operacors  indicated  that  two  different  voice 
messages  incoming  at  the  same  time  on  the  H-161  headset  Interfered 
with  reception.  Estimated  percentages  of  the  time  this  occurred 
varied  from  0  to  25-30  percent.  However,  only  two  operators  rated 
understanding  voice  communications  as  being  less  than  "easy." 

°  Six  of  the  eight  operators  reported  they  could  not  read  the  output 
of  the  audit  printer  without  moving  or  lifting  the  paper.  Four  of 
the  operators  indicated  they  had  to  stand  up  to  read  the  printed 
pages . 

°  Five  of  the  six  operators  responding  reported  that  they  either 
"seldom"  or  "practically  never"  heard  the  alarm  on  the  DCT  when 
critical  or  flash  messages  were  received.  (Data  obtained  from  the 
post-trial  interviews  indicated  that  the  alarm  was  not  loud  enough 
to  be  heard  when  wearing  the  H-161  headset.) 

°  Five  of  the  eight  operators  reporting  indicated  that  lighting  for 
making  the  red  patch  panel  connections  was  less  than  adequate. 
Lighting  for  other  purposes  was  judged  to  be  either  "adequate"  or 
"very  adequate." 

°  Storage  space  for  personal  papers  and  items  was  rated  as  "very 
inadequate”  by  five  of  the  eight  operators.  Only  one  of  the 
operators  rated  the  space  as  being  "adequate"  or  better.  Four  of 
the  operators  rated  legroom  as  being  less  than  adequate,  and  five 
of  the  eight  rated  elbow  room  as  being  less  than  adequate. 

°  No  significant  injuries  to  personnel  were  reported.  However,  five 
of  the  operators  indicated  they  had  experienced  pinched  or  mashed 
fingers  or  hands  during  the  trials.  No  more  than  three  of  the 
operators  reported  any  physical  symptoms  of  consequence  on  a  list 
of  10  symptoms  provided. 

°  Six  of  the  eight  operators  reported  that  it  was  less  than  easy  to 
load/key  the  crypto  devices. 

°  FSIC  operators  were  asked  to  rate  the  adequacy  of  system  prompts 
for  performing  each  of  11  listed  operations.  Prompts  for  nine  of 
the  11  were  rated  as  being  adequate  by  a  minimum  of  five  of  the 
operators.  Three  operators  did  not  respond  to  "Entering  office 
and  operator  names"  and  two  did  not  respond  to  "Entering 
password" . 

°  Seven  of  the  eight  operators  indicated  they  had  no  problems  in 
shutting  down  the  system  and  preparing  for  system  movement. 
Selecting  a  site  and  leveling  the  vehicle  were  also  reported  to  be 
"easy"  or  "very  easy." 

°  All  eight  operators  indicated  that  the  Altroute  function  worked 
"well"  or  "very  well"  and  that  Altrouting  was  either  "easy"  or 
"very  easy." 
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°  FS1C  operators  were  asked  to  rate  the  ease  of  performing  each  of 
13  tasks.  Four  or  more  operators  reported  that  message  dissemina¬ 
tion,  transcribing  messages  from  voice  to  text,  and  correcting 
message  error  formats  was  less  than  easy. 

°  Seven  of  20  listed  commands  were  not  used  by  any  of  the  eight 
operators.  Only  one  of  the  commands,  line  message  logging  (LML), 
was  used  by  all  eight  operators. 

°  Of  eight  listed  recall  modes,  only  one  was  used  by  more  than  two 
operators,  three  were  not  used  at  all,  and  three  were  used  by  only 
one  operator  each. 

°  Three  of  the  six  operators  reporting  indicated  that  it  was 
"difficult"  or  “very  difficult”  to  understand  the  rules  for 
determining  addresses  for  ACTION  and  INFO  messages.  The  other 
three  operators  rated  it  as  "in-between." 

°  Operators  were  asked  to  estimate  what  percentage  of  their 
operating  time  they  spent  seeking  help  from  sources  such  as 
contractor  personnel,  the  HLP  menu,  manuals,  class  notes  and  other 
operators.  Three  operators  Indicated  they  spent  30-40  percent  of 
their  time,  two  indicated  20-30  percent ,  two  indicated  10-20 
percent,  and  one  Indicated  10  percent  or  less.  Five  operators 
Indicated  they  never  used  the  HLP  command  with  one  operator 
indicating  the  help  provided  was  adequate,  while  two  of  those  who 
used  it  rated  it  as  "in  between."  (During  the  interviews  it  was 
discovered  that  two  of  the  operators  did  not  know  that  the  HLP 
command  existed.) 

°  All  eight  operators  indicated  that  they  used  checklists  at  some 
time  while  performing  their  jobs,  with  seven  of  the  eight 
indicating  they  used  them  for  initializing  the  equipment. 

°  Seven  of  the  eight  operators  reported  that  they  received  at  least 
one  error  message,  fatal  message,  informational  message,  or 
unusual  occurrence  message  that  they  did  not  understand.  Five  of 
these  reported  that  the  average  time  required  to  understand  these 
messages  was  more  than  five  minutes. 

°  Of  the  seven  status  displays  or  statistical  reports,  only  the 
queue  statistics  report  and  CSP  dynamic  status  display  was  rated 
as  being  of  at  least  moderate  use  by  more  than  half  of  the 
operators . 

°  All  eight  of  the  operators  rated  the  readability  of  the  GRIDSE-T 
displays  as  being  either  "easy"  or  "very  easy."  However,  three  of 
the  operators  rated  the  readability  of  numerals,  letters,  and/or 
commands  on  the  GRIDSE-T  keyboard  as  being  either  "difficult"  or 
"very  difficult."  (It  was  discovered  that  one  of  the  FSIC 
keyboards  had  paste-on  key  designators  which  had  become  worn  and 
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were  difficult  to  read.)  All  seven  of  the  operators  reporting 
Indicated  that  it  would  be  either  “difficult”  or  “very  difficult” 
to  use  the  GRIDSE-T  keyboard  while  wearing  MOPP  IV  gear. 

°  Seven  of  the  eight  operators  did  not  feel  that  operating  the  FSIC 
placed  too  large  a  memory  burden  on  the  operator. 

°  Five  of  the  eight  operators  reported  that  their  performance  of 
some  tasks  became  less  effective  during  peak  workload  times  and 
also  became  unsatisfactory. 

°  Operators  were  asked  to  rate  the  FSIC's  performance  in  five 

functions  in  providing  an  interface  between  the  sensors  and  the  MI 
Battalion  TCAE.  The  results  are  presented  in  Table  1. 

0  The  communications  capability  and  the  ability  to  deal  with  large 
volumes  of  data  were  seen  by  all  of  the  operators  as  the  best 
points  about  the  system.  Six  of  the  operators  reported  that 
complexity  was  the  major  problem  area.  Two  operators  provided 
written  comments  indicating  that  operator  workload  was  too  great. 

o  Emplacing  and  leveling  the  FSIC  vehicle  was  seen  as  being  “easy" 
or  “very  easy"  by  all  seven  operators  who  responded. 

o  Two  of  the  CEWI  FSIC  operators  rated  their  pretrial  understanding 
of  the  tasks  required  of  them  as  being  either  “good"  or  “very 
good,"  while  all  four  rated  their  posttrial  understanding  as 
either  “good”  or  "very  good."  A  tendency  in  the  opposite  direc¬ 
tion  was  found  among  the  forward  FSIC  operators.  One  of  the  two 
operators  who  rated  his  pretrial  understanding  as  being  "in-bet¬ 
ween"  rated  his  posttrial  understanding  as  "very  poor." 


Table  1 


Ratings  of  FSIC  Performance  In  Providing  an  Interface  Between  the  Sensors 
and  the  MI  Battalion  TCAE 


Adequate 
or  better 

Inadequate 
or  worse 

No 

response 

Providing  data  communications  between 
sensors  and  the  TCAE,  and  converting 
messages  into  common  formats. 

6 

2 

- 

Providing  a  capability  for  transcribing 
voice  messages  into  automated  message 
formats. 

5 

2 

1 

Providing  data  communications  between 
the  MI  Battalion  TCAE  and  the  sensors. 

4 

4 

- 

Assisting  in  the  identification  saniti¬ 
zation  and  routing  of  combat  informatidn 
to  the  Brigade. 

3 

4 

1 

Providing  automated  interface  with  the  AIM.  5 

1 

2 

Note.  Each  topic  was  rated  by  all  8  FSIC  operators. 

Discussion  of  FSIC  operator  findings.  Based  on  operator  reports,  the 
FSIC  did  not  present  any  apparent  real  health  or  safety  hazards  to  personnel. 
Displacement  and  emplacement  of  the  system  were  also  seen  as  being  free  of 
significant  problems.  The  ability  of  the  system  to  handle  large  volumes  of 
data  was  seen  as  a  major  plus  for  the  system.  Only  two  of  the  more  frequently 
performed  tasks  were  rated  as  being  "difficult"  or  "very  difficult"  by  two  or 
more  of  the  operators  responding. 

There  are  several  indications  that  the  system  was  not  fully  exercised 
during  the  trials.  While  this  may  have  been  due  in  part  to  the  scenarios 
selected,  it  is  more  likely  indicative  of  a  lack  of  familiarity  with  the 
system  on  the  part  of  the  operators.  For  example,  the  operators  felt  that 
they  should  have  had  some  training  on  the  cryptographic  devices  employed  in 
the  FSIC.  They  also  spent  considerable  time  seeking  help  in  one  form  or 
another.  Five  of  the  eight  reported  that  they  spent  20  percent  or  more  of 
their  time  in  seeking  help,  and  all  indicated  they  used  checklists  at  some 
time  or  another.  A  lack  of  familiarity  with  some  of  system  outputs  further 
suggests  that  the  operators  were  not  fully  trained  or  practiced.  Seven  of  the 
eight  reported  that  they  received  at  least  one  system  message  that  they  did 
not  understand,  and  five  of  these  reported  that  the  average  time  required  to 
understand  one  of  these  messages  was  more  than  five  minutes.  Since  the 
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operators  did  not  feel  that  operating  the  FSIC  placed  too  large  a  memory 
burden  on  the  operator,  the  seemingly  excessive  time  spent  seeking  help  and 
failure  to  understand  messages  reflect  a  need  for  further  training  and 
experience.  A  final  bit  of  evidence  comes  from  the  lack  of  utilization  of  the 
system's  capabilities.  Seven  of  20  presumably  common  commands  were  not  used 
by  any  of  the  operators,  with  only  one  command  used  by  all  eight.  Similarly, 
only  one  of  eight  recall  modes  was  used  by  more  than  two  operators  while  three 
were  not  used  at  all.  The  extent  to  which  the  utilization,  or  lack  thereof, 
was  driven  by  the  scenarios,  and  which  was  driven  by  operator  familiarity  and 
comfort  in  using  only  a  few  commands,  was  not  determined. 

There  were  a  few  problems  which  surfaced  which  are  seen  as  deficiencies 
that  should  be  corrected.  One  of  these  concerns  the  DOT.  Five  of  the  six 
operators  responding  reported  that  they  either  "seldom”  or  "practically  never" 
heard  the  alarm  when  critical  or  flash  messages  were  received.  Posttrial 
interviews  indicated  that  the  audible  alarm  was  too  weak  to  be  heard  while  the 
operators  were  wearing  their  headsets.  A  flashing  light  placed  between  the 
keyboard  and  the  screen  on  the  GRIDSE-T  or  an  audible  alarm  wired  directly 
into  the  headset  should  alleviate  this  problem  without  any  major  modifications 
to  the  equipment. 

As  the  system  is  now  designed,  two  unrelated  voice  messages  can  be 
received  simultaneously,  with  each  message  going  to  a  different  ear.  Some 
operators  indicated  that  this  interference  occurred  from  as  much  as  25  to  30 
percent  of  the  time.  This  arrangement  is  not  satisfactory  as  neither  of  the 
simultaneous  messages  is  likely  to  be  understood.  The  operator  will  have  to 
call  back  to  each  party,  and  precious  time  will  be  lost.  During  the  later 
interviews,  the  operators  indicated  that  they  prevented  the  interference  to 
the  extent  possible  by  removing  the  opposite  earphone  whenever  a  message  was 
being  received.  This  could,  of  course,  cause  the  second  message  to  be 
completely  lost.  It  is  recommended  that  the  system  be  redesigned  so  that  a 
second  and  any  subsequent  messages  be  held  until  the  first  message  is  com¬ 
pleted.  This  could  be  done  either  by  having  a  "busy  signal"  at  the  sender's 
end,  a  "message  waiting"  light  at  the  receiver's  end,  or,  having  all  subsequent 
messages  recorded  for  review  in  sequence  by  the  operator. 

The  operators  reported  that  they  had  extreme  difficulty  in  using  the 
GRIDSE-T  keyboard  while  wearing  MOPP  IV  gloves.  If  it  is  anticipated  that 
this  circumstance  would  ever  occur,  a  tool  approximately  the  size  of  a  pencil, 
but  one  which  could  be  used  for  virtually  no  other  purpose  so  as  to  prevent 
loss,  should  be  placed  in  a  rack  or  some  conspicuous  place  where  it  can  be 
readily  retrieved  by  the  operator  if  needed.  It  has  been  suggested  that 
consideration  be  given  to  redesigning  the  keyboard  layouts  to  better  support 
their  use  when  wearing  MOPP  gloves. 

Storage  space  for  personal  papers  and  personal  items  was  rated  as  "very 
inadequate"  by  five  of  the  eight  operators.  The  lack  of  space  for  storage  is 
obvious  even  to  the  most  casual  observer.  Much  of  the  electronic  gear  in  the 
shelter  is  off-the-shelf  equipment  and  is  far  from  being  state  of  the  art.  If 
newer  equipment  is  obtained  as  replacements,  this  smaller  equipment  might 
alleviate  the  storage  space  problem.  If  such  is  not  anticipated,  some 
additional  storage  space  should  be  considered.  For  example,  storage  boxes  on 


'iVWlfBI 


isasw 


the  side  of  the  air  conditioner  over  the  cab  might  go  a  long  way  to  alleviating 
the  problem.  Replacement  of  the  bulky  equipment  with  state-of-the-art 
equipment  should  also  reduce  the  reported  legroom  problem. 

Six  of  the  eight  operators  reported  that  they  could  not  read  the  output 
of  the  audit  printer  without  moving  or  lifting  the  paper.  Four  of  these 
indicated  that  they  had  to  stand  to  read  the  output.  At  first  glance  this 
might  seem  like  a  major  problem.  However,  during  the  interviews  after  the 
trials,  none  of  the  operators  felt  that  these  facts  had  a  significant  impact  on 
their  performance. 

AIM  operator  questionnaire  findings.  Responses  to  some  items  which  met 
the  criteria  (i.e.,  75  percent  positive  or  25  percent  negative)  are  not 
included  in  the  summary  because  of  the  large  numbers  of  nonrespondents  to  the 
items.  It  was  felt  that  this  might  bias  the  sample  in  some  way.  Also, 
responses  to  some  items  are  included  that  did  not  meet  the  criteria  because  of 
the  importance  of  the  function  or  task  in  question.  There  were  a  total  of  28 
AIM  operators,  not  all  of  whom  answered  every  questionnaire  item.  The  findings 
are  shown  in  bullet  format  below. 


°  Both  operator  and  system  supervisors  were  asked  to  recall  how  well  they 
felt  they  understood  the  tasks  required  of  them  when  they  completed 
training.  They  were  also  asked  t6  rate  how  well  they  felt  they  understood 
them  at  the  time  the  questionnaires  were  administered  (item  1).  There  was 
a  tendency  for  the  personnel  to  rate  their  understanding  lower  at  the  time 
the  questionnaires  were  administered,  although  the  difference  did  not 
reach  statistical  significance.  Table  2  shows  the  responses  to  these  two 
items. 

Table  2 


Ratings  of  Understanding  of  Tasks  Required  at  the  Completion  of  Training  and 
at  the  Time  the  Questionnaires  were  Completed 


Understanding  of 
tasks  required 

At  time  of 
Completion 
of  training 

At  time  of 

Completion  of 
questionnaire 

Good  or 

very  good 

22 

16 

In-between 

poor  or  very  poor 

6 

12 

Note.  The  above  ratings 

are  from  all 

28  AIM  operators. 

15 


°  Over  two-thirds  of  the  AIM  personnel  responding  felt  that 
onloading,  offloading,  and  moving  the  MECAs  was  "easy"  or  “very 
easy.**  However,  there  was  concern  over  safety  for  some  of  the 
operations.  Of  those  personnel  responding,  29  percent  reported 
they  felt  “unsafe**  or  “very  unsafe”  while  unloading  the  MECAs  from 
the  vehicle,  28  percent  reported  they  felt  “unsafe”  or  "very 
unsafe"  while  attaching  wheels  and  handles  to  the  MECAs,  and  30 
percent  reported  they  felt  "unsafe”  or  very  unsafe"  while  loading 
the  MECAs  on  the  transport  vehicle.  The  reports  on  safety  were 
made  based  on  experiences  during  training  at  the  contractor 
facility. 

°  Twenty-three  of  the  27  AIM  personnel  responding  indicated  that 
they  used  checklists  when  powering  up  the  AIM  shelter  components 
at  some  time  during  the  trials. 

°  All  28  of  the  AIM  personnel  indicated  that  lighting  was  adequate 
in  the  shelter  for  powering  up  and  powering  down  the  AIM  system. 

°  Twenty-two  of  the  28  AIM  personnel  reported  that  all  cables  and 
connectors  were  adequately  labeled  or  coded  to  ensure  proper 
connection  to  the  MECAs  without  damage  to  the  equipment  or  injury 
to  the  personnel.  All  27  respondents  indicated  adequate  labeling 
for  netting  shelter  cables  and  connectors. 

°  Twenty  of  the  27  operators  responding  indicated  that  they  did  not 
use  the  map  displays  on  their  consoles.  Those  who  responded  that 
they  did  use  the  maps  were  requested  to  answer  several  questions 
concerning  the  overall  adequacy  of  the  maps.  However,  a  number  of 
the  personnel  who  Indicated  they  did  not  use  the  maps  also 
responded  to  the  items.  The  responses  of  14  operator  personnel 
who  provided  answers  to  these  questions  are  shown  in  Table  3. 


Table  3 

Responses  to  Items  Concerning  Map  Displays  at  AIM  Workstations 


Rating 

Level 

Reading 

Letters 

Area 

Coverage 

Placing 

Symbols 

Deleting 

Symbols 

Distinguishing 

Symbols 

5  &  4 

7 

5 

6 

6 

3 

3 

5 

2 

2 

2 

6 

2  &  1 

2 

7 

6 

6 

5 

Note  1.  Rating  level  5  means  "Very  easy”  or  "Very  adequate."  Rating  level  4 
means  "Easy”  or  ‘'Adequate.'*  Rating  level  3  means  "Marginal"  or  "In  between." 
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Rating  level  2  means  “Difficult”  or  "Inadequate."  Rating  level  1  means  "Very 
difficult"  or  "Very  inadequate." 

Note  2.  14  AIM  operators  rated  each  of  the  five  items. 

°  DTOC  AIM  operators  and  supervisors  were  provided  a  list  of  36 
GESCAN  operations  and  asked  to  rate  the  ease  with  which  they 
could  be  performed.  Fourteen  of  these  functions  were  performed 
by  less  than  half  of  the  DTOC  personnel.  Based  on  a  rating 
scale  of  very  easy  ■  1  to  very  difficult  »  5,  the  easiest  to 
perform  functions  were  obtaining  Hit  Files  for  browsing  (1.91), 
deleting  Hit  Files  (1.92),  aborting  queries  (1.92),  copying 
queries  (2.00),  deleting  queries  (2.00),  printing  queries 
(2.00),  and  running  queries  (2.00).  The  most  difficult  functions 
to  perform  were  editing  queries  (3.40),  deciding  where  additional 
terms  should  be  added  to  queries  (3.14),  determining  fields  for 
query  terms  (3.06),  and  choosing  terms  for  queries  (3.03). 

°  MICROFIX  workstation  operators  rated  the  ease  or  difficulty  of 
performing  each  of  10  terminal  control  processor  operations. 

More  than  half  of  those  responding  rated  all  as  being  "easy," 
with  75  percent  or  more  rating  five  of  the  operations  as  being 
"easy."  Only  three  operations  received  any  "difficult"  ratings. 

°  AIM  personnel  were  asked  to  report  if  they  noticed  any  differen¬ 
ces  in  system  response  time  under  four  different  conditions. 

Table  4  presents  the  responses  of  those  completing  the  item. 


Table  4 

Ratings  of  Differences  in  System  Response  Times 


Yes 

No 

Time  of  day 

0 

24 

Number  of  stations  operating 

15 

10 

Intensity  of  the  situation 

13 

11 

Type  of  Requirement 

15 

9 

Note.  24,  25,  24,  and  24  of  the  28  AIM  operators  and  supervisors  responded  to 
the  above  four  items. 

°  Eleven  of  the  19  MICROFIX  operators  responding  reported  that 
response  time  to  commands  at  their  stations  was  "slow"  or  "very 
slow,"  six  rated  response  times  as  "in-between,”  and  two  rated  it 
as  "fast"  or  "very  fast." 
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°  Seven  of  19  MICROFIX  operators  reported  that  response  time  to 
tasking  messages  was  "unsatisfactory"  or  "very  unsatisfactory" 
during  normal  periods  of  operation,  11  gave  these  same  response 
when  queried  about  response  time  during  peak  periods  of  operation. 

°  Twelve  of  22  AIM  operators  responding  stated  that  the  organization 
of  information  displayed  was  not  helpful  for  doing  their  jobs. 

0  Sixteen  of  the  26  AIM  personnel  responding  reported  that  generat¬ 
ing  MISMART  messages  was  either  "easy"  or  “very  easy."  Three 
reported  it  as  being  "difficult"  or  "very  difficult." 

o  AIM  personnel  were  asked  to  report  on  any  symptoms  they  developed 
while  operating  at  their  stations.  They  were  asked  to  indicate 
how  long  they  had  been  on  their  shifts  when  each  of  the  symptoms 
checked  occurred.  The  times  reported  were  highly  variable,  with 
some  respondents  indicating  a  symptom  occurred  in  less  than  an 
hour  after  being  on  the  shift,  while  others  indicated  the  same 
symptom  did  not  occur  until  several  hours  into  the  shift.  The 
responses  to  the  symptoms  developed  are  shown  in  Table  6. 


Table  5 

Reported  Physical  Symptoms  Developed  During  AIM(6)  Operations 


DTOC  Personnel 
Yes 

No 

TCAE 

Yes 

Personnel 

No 

Eyestrain 

9 

5 

8 

3 

Blurred  vision 

3 

12 

3 

8 

Headaches 

5 

9 

5 

6 

Dizziness 

1 

14 

1 

10 

Fatigue 

9 

6 

7 

4 

Neck  &  shoulder  stiffness 

13 

2 

5 

6 

General  muscle  stiffness 

9 

6 

5 

6 

Finger  fatigue 

2 

13 

1 

10 

Backache 

10 

5 

5 

6 

Hearing  difficulties 

0 

15 

0 

9 

Note .  Two  TCAE  supervisors  did  not  respond.  One  DTOC  operator  did  not 
respond  to  the  "eyestrain"  symptom,  and  one  did  not  respond  to  "headaches." 
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°  Twenty-four  of  27  AIM  personnel  responding  reported  that  it  would 
be  either  "difficult"  or  “very  difficult"  to  prepare  the  AIM 
stations  while  wearing  MOPP  IV  gear,  and  21  of  the  27  respondents 
indicated  that  it  would  be  either  “difficult"  or  “very  difficult" 
to  use  the  keyboard  while  wearing  MOPP  IV  gear. 

°  Tables  6  and  7  show  the  responses  of  the  28  AIM(6)  personnel  to 
questions  concerning  the  strengths  and  weaknesses  of  the  system. 
Respondents  were  encouraged  to  check  as  many  of  the  alternatives 
as  they  felt  applicable.  They  were  also  encouraged  to  provide 
written  comments  to  specify  other  strengths  and  weaknesses.  The 
written  comments  concerning  problems  (Table  6)  typically  indicated 
that  the  system  did  not  do  what  they  needed  it  to  do.  Written 
comments  concerning  the  best  points  typically  indicated  that  the 
concept  behind  the  system  was  good. 


Table  6 

Responses  to  the  Question  "What  are  the  Major  Problem  Areas  of  the  System?” 


Problem  Area 


Number  of  Responses 


Complexity  10 
Lack  of  software  to  support  functions  14 
Poor  hardware  design  11 
Poor  software  design  15 


Note.  28  AIM  operators  had  the  opportunity  to  answer  the  four  items. 
Table  7 

Responses  to  the  Question  "What  are  the  Best  Points  About  the  System?" 


Best  Points 

Number  of  Responses 

Communications  capability 

9 

Ability  to  deal  with  large  volumes  of  data 

13 

Speed  of  response 

2 

Ease  of  operator  interface 

5 

Note.  28  AIM  operators  had  the  opportunity  to  answer  the  above  items. 
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°  Twenty-five  of  27  respondents  did  not  know  when  they  should  "bump 
start"  the  ECU  in  the  AIM  shelter. 

°  Eleven  of  26  respondents  felt  that  the  system  placed  too  large  a 
memory  burden  on  the  operator. 


Discussion  of  AIM  operator  responses.  As  was  the  case  with  the  FS1C 
operators,  there  are  indications  that  the  training  and  practice  received  prior 
to  the  trials  were  somewhat  lacking.  There  was  a  tendency  for  AIM  operators 
to  rate  their  understanding  of  the  tasks  required  of  them  somewhat  lower 
during  the  trials  than  at  the  completion  of  their  training  period.  Some 
operators  indicated  they  continued  to  use  checklists  throughout  the  trials, 
although  approximately  half  reported  they  used  them  only  until  they  became 
familiar  with  the  procedures.  Either  because  the  scenarios  did  not  fully 
exercise  the  system,  or  the  operators  were  not  familiar  with  all  the  func¬ 
tions,  14  of  36  GESCAN  functions  were  performed  by  less  than  half  of  the 
personnel.  Development  of  queries,  in  particular,  seemed  to  present  problems. 

It  must  be  remembered  that  the  operator  stations  involved  in  the  trials 
were  substitutes  for  the  PAWS  which  is  intended  to  be  part  of  the  objective 
ASAS  system.  Also,  only  a  very  preliminary  version  of  the  software  which  will 
become  available  was  available  during  the  trials.  Therefore,  the  pattern  and 
extent  of  weaknesses  or  deficiencies  reported  by  the  operators  may  be  quite 
different  when  the  next  generation  of  equipment  is  received,  tested,  and 
reported  upon.  Nevertheless,  note  should  be  taken  of  deficiencies  reported  to 
ensure  that  they  are  removed  from  the  eventual  system. 

The  major  pluses  reported  by  the  operators  were  the  communications 
capability  and  the  ability  for  the  system  to  handle  large  volumes  of  data. 
Terminal  control  processor  functions  were  considered  generally  to  be  easy  to 
perform.  The  AIM  shelter,  which  is  undoubtedly  similar  to  the  eventual  system 
shelter  and  components,  did  not  seem  to  create  any  problems  for  the  operators. 
Lighting  and  labeling  were  adequate,  and  no  problems  in  energizing  or  shutting 
down  the  system  were  reported. 

Ratings  of  speed  of  response  for  the  system  were  generally  very  poor. 

One  of  the  contractor  personnel  stated  that  this  was  due  in  large  part  to  a 
small  and  relatively  slow  computer  Jn  the  MICROFIX  stations.  It  is  expected 
that  speed  of  response  will  be  improved  with  the  new  equipment.  Support  of 
both  DTOC  and  TCAE  functions  received  generally  poor  ratings.  It  is  an¬ 
ticipated  that  software  development  will  greatly  improve  the  system's 
capability  in  these  areas. 

During  the  trials,  the  MECAs  were  kept  on  an  expandable  five-ton  van 
rather  than  being  carried  on  a  CUCV  during  displacement/emplacement  as  they 
were  during  training.  The  operators  had  concern  about  safety  during  emplacem¬ 
ent/displacement  practice  while  training  at  the  contractor's  facility. 

However,  the  PAWS  is  expected  to  weigh  several  hundred  pounds  less  than  the 
current  MECAs,  so  future  concern  for  safety  while  handling  the  lighter 
equipment  is  likely  to  be  considerably  reduced. 
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AIM  personnel  reported  many  more  physical  symptoms  than  did  the  FSIC 
personnel.  Eyestrain,  fatigue,  neck  and  shoulder  stiffness,  and  backache  were 
reported  by  more  than  half  of  the  AIM  personnel.  In  contrast,  none  of  the 
FSIC  personnel  reported  eyestrain,  although  three  of  the  eight  FSIC  operators 
did  report  general  fatigue  and  backache.  The  fatigue  and  backache  undoubtedly 
stem  from  long  hours  sitting  In  front  of  a  computer  terminal.  The  difference 
In  reported  eyestrain  Is  difficult  to  understand  as  Is  the  generally  higher 
level  of  symptoms  reported  by  the  AIM  personnel.  No  apparent  reason  for  this 
could  be  discerned,  nor  did  the  later  interviews  add  any  understanding  of 
this  subject. 

There  were  two  concerns  reported  by  AIM  operators  which  should  be 
considered  in  developing  the  PAWS.  One  of  these  should  create  no  real 
problem.  AIM  operators  reported  that  they  would  have  difficulties  in  using 
the  terminal  keyboard  while  wearing  MOPP  IV  gear.  Again,  a  pencil  sized  tool, 
that  would  have  no  other  purpose,  should  be  added  to  a  built-in  bracket  on  the 
PAWS.  This  would  permit  the  operator  to  use  the  keyboard,  albeit  more  slowly. 
It  has  been  suggested  that  consideration  could  be  given  to  redesigning  the 
keyboards  to  facilitate  their  use  by  MOPP-glove-wearing  personnel. 

The  second  concern  expressed  relates  to  map  area  sufficiency.  Twelve  of 
the  18  MICROFIX  station  operators  responding  reported  they  did  not  use  the 
maps  available  on  their  displays.  In  all,  20  of  the  AIM  personnel  reported 
maps  were  not  used.  Since  the  TCAE  personnel  do  not  do  order  of  battle,  the 
results  from  the  DTOC  are  more  pertinent.  Half  of  the  DTOC  MICROFIX  operators 
reported  they  did  not  use  the  maps,  and  one  who  did  reported  that  it  was  only 
for  a  quick  familiarization.  Based  on  the  comments  received,  the  problem  was 
that  the  territory  covered  at  any  one  time  was  too  small.  The  operators  had 
the  capability  of  scanning  the  map  area,  but  the  total  area  that  was  available 
on  the  CRT  at  any  one  time  was  too  small  to  give  them  the  "big  picture." 

Operator  Interview  Findings 

Administration  conditions.  With  the  exception  of  two  FSIC  operators  and 
one  AIM  operator  who  were  interviewed  individually,  all  other  personnel  were 
interviewed  in  groups  of  two  to  five.  This  was  done  for  two  reasons.  First, 
other  personnel  in  a  group  could  serve  as  a  check  on  the  accuracy  of  state¬ 
ments  made  by  one  individual,  provided  they  too  performed  the  functions  or 
tasks  in  question.  Second,  the  limited  amount  of  time  available  during 
breaks  in  operations  due  to  system  malfunctions  and  other  reasons  was  not 
adequate  to  permit  lengthy  individual  interviews  during  the  trials.  Interview 
times  ranged  between  one  and  two  hours  for  the  groups,  and  somewhat  less  than 
an  hour  for  the  individuals.  A  set  of  prepared  questions  was  employed  to 
guide  the  interviews  (see  Appendixes  C  and  D),  but  interviewees  were  en¬ 
couraged  to  provide  any  additional  information  they  felt  to  be  relevant. 

It  was  assumed  that  statements  made  during  the  interviews  represented  the 
opinions/attitudes  of  all  members  of  a  group  unless  there  was  disagreement. 
Disagreements  were  rare,  but  other  individuals  in  a  group  frequently  clarified 
or  added  to  statements  made  by  another  individual.  Therefore,  any  statement, 
as  amended  or  expanded  during  the  interview,  was  considered  to  represent  a 
consensus  for  reporting  purposes. 


21 


FSIC  operator  Interview  findings.  These  are  presented  below  In  bullet 
format. 

°  All  FSIC  operators  agreed  that  the  workload  of  the  forward  FSIC 
operators  was  higher  than  that  of  the  CEUI  FSIC  operators. 
Reasons  cited  were  the  requirement  to  send  messages  to  Brigade 
and  to  send  and  receive  messages  on  the  AN/PVS-2.  No  mention  was 
made  of  the  requirement  for  forward  FSIC  operators  to  correct 
format  errors  in  messages. 

°  None  of  the  operators  felt  that  they  had  problems  of  any  conse¬ 
quence  with  the  generators. 

°  All  of  the  operators  felt  that  the  training  they  received  on  the 
FSIC  was  inadequate.  Estimates  of  training  time  required  to 

produce  effective  operators  varied  from  four  weeks  to  one  year. 
The  need  for  basic  computer  training  was  expressed  by  all. 
Training  in  typing  skills,  for  those  who  had  not  already  acquired 
them,  was  also  felt  to  be  necessary. 

°  The  operators  felt  that  they  were  not  using  their  MOS  skills, 
especially  in  the  CEUI  FSIC.  Several  respondents  stated  that  the 
FSIC  operator  should  have  a  communications  MOS.  Two  groups 
mentioned  that  an  analyst  equipped  with  a  DCT  and  a  radio  should 
be  put  in  another  forward  location,  thus  allowing  the  FSIC  to 
function  only  as  a  communications  device. 

°  Forward  FSIC  operators  had  great  difficulty  in  hearing  the  alarm 
on  the  DCT. 

°  The  first  echelon  operator  maintenance  manuals  did  not  serve  their 
intended  purpose.  The  index  was  judged  to  be  poor  and  the 
information  needed  when  a  problem  occurred  was  either  not  in  the 
manuals  or  could  not  be  found. 

°  The  procedural  checklists  provided  by  the  contractor  were  rated 
very  highly. 

°  Operators  had  difficulty  in  judging  whether  or  not  the  work  they 
were  doing  required  more,  the  same,  or  fewer  personnel  than 
required  when  using  manual  methods.  This  was  because  the  tasks 
they  were  performing  were  not  the  same  as  those  they  performed 
previously. 

0  None  of  the  operators  felt  that  the  job  created  any  unusual 
stresses,  but  two  mentioned  that  persons  with  claustrophobic 
tendencies  should  not  be  placed  in  the  FSIC. 

°  The  FSIC  operating  system  was  not  considered  to  be  user  friendly. 
Initializing  was  stated  to  be  too  time  consuming,  requiring  too 
many  commands,  and  requiring  the  same  information  to  be  entered 
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more  than  once.  Also,  the  system  provided  little  help  in 
correcting  errors  or  in  helping  the  operator  in  case  of  a  software 
malfunction.  Error  messages  were  considered  to  be  insufficiently 
explanatory,  and  most  operators  simply  called  the  contractor 
personnel  any  time  a  problem  occurred. 

°  Operators  mentioned  that  attaching  the  UHF  antenna  could  be 
hazardous  in  icy  conditions  as  it  would  typically  require  the 
operator  to  climb  to  the  top  of  the  shelter.  The  operators  felt 
that  there  were  no  other  health  or  safety  hazards  inherent  in  the 
system. 

Discussion  of  FSIC  operator  interviews.  The  interviews  corroborated  and 
clarified  some  of  the  responses  obtained  to  the  questionnaires.  Of  particular 
note  is  the  finding  that  the  operators  unanimously  felt  that  basic  training  in 
computer  systems  and  computer  operations  were  necessities.  They  claimed  that 
they  did  not  understand  what  they  were  doing  on  occasion.  That  is,  they  did 
not  know  why  certain  inputs  were  required,  and  generally  operated  by  strictly 
rote  procedures.  This  may  be  part  of  the  reason  they  felt  that  the  FSIC  was 
not  user  friendly  and  why  they  had  to  seek  help  so  often.  The  implications 
for  training  are  fairly  clear.  Some  computer  basics  need  to  be  included  in 
training,  as  well  as  more  specifics  on  the  purpose  and  function  of  various 
inputs  on  the  FSIC  system. 

None  of  the  operators  had  any  praise  for  the  operator  maintenance 
manuals.  This  may  have  resulted  in  part  from  the  operators'  lack  of  knowledge 
concerning  computers.  Nevertheless,  there  appears  to  be  a  need  for  revision, 
with  an  improved  and  more  comprehensive  index.  Also,  training  in  the  use  of 
the  manuals  should  be  a  part  of  the  program.  Manuals  were  not  available  to 
the  operators  during  the  original  training  at  the  contractor's  facility. 

The  need  for  a  more  suitable  alarm  for  the  DCT  was  corroborated  during 
the  interviews.  Since  the  alarm  is  there  to  alert  the  operator  of  the  arrival 
of  extremely  Important  messages,  it  is  imperative  that  some  other  alerting 
system,  such  as  the  audible  beeps  suggested  earlier,  be  designed  into  the 
system. 

The  FSIC  operators  felt  that  their  MOS  skills  as  analysts  were  only  very 
minimally  utilized.  They  suggested  that  a  separate  MOS,  with  less  analytical 
and  more  communications  training,  be  initiated  for  the  FSIC.  Also,  the  merit 
of  separating  the  analytical  and  communications  functions  completely,  as 
suggested  by  forward  FSIC  operators,  should  be  examined. 

Although  no  injuries  of  consequence  were  reported  during  the  trials, 
operators  felt  that  climbing  on  top  of  the  shelter  to  emplace  the  UHF  antenna 
in  icy  conditions  could  be  extremely  hazardous.  Emplacement  of  the  antenna 
without  mounting  the  shelter  was  viewed  as  being  extremely  difficult  for 
average  sized  soldiers.  Also,  one  group  expressed  fear  of  turning  the  vehicle 
over  when  traversing  rough  terrain  due  to  the  top  heavy  load  in  the  shelter. 


Ocher  consents  of  interest  which  are  worthy  of  consideration  in  design, 
personnel  selection  and  training  were: 

o  Indicators  on  much  of  the  equipment  are  difficult  to  see  from  the 
operator's  position, 
o  More  room  for  writing  is  needed. 

o  Controls  on  the  multicoupler,  one  radio  and  the  CFP  are  hard  to  reach, 
o  The  equipment  must  be  shut  down  to  switch  generators.  It  takes  at 
least  20  minutes  to  get  back  on  the  air. 
o  The  message  edit  menu  is  not  tailored  to  the  operator's  needs, 
o  Operators  must  either  have  or  be  taught  typing  skills  in  training, 
o  The  battle  override  switch  can  be  kicked  by  accident, 
o  A  speaker  should  be  provided  in  the  cab  to  give  relief  from  the 
headset  during  movement. 

o  Personnel  with  claustrophobic  tendencies  should  be  screened  out. 

AIM  operator  interviews.  These  findings  are  presented  below  in  bullet 
format. 

°  The  operators  as  a  group  felt  that  the  system  functioned  too 
slowly.  Specific  examples  given  were  the  time  to  download  a  data 
base  to  the  MICROFIX  (as  much  as  20  minutes),  having  to  send 
several  messages  to  transmit  a  single  message  because  the  formats 
were  too  short,  the  ability  to'  run  only  one  query  at  a  time,  and 
the  inability  to  skip  steps  in  a  procedure. 

°  Cabling  the  system  in  the  field  was  seen  as  a  problem  by  the  TCAE 
personnel.  Labels  were  reported  to  get  dirty  or  lost,  and  that 
the  number  was  excessive  (10  cables  on  the  generator).  Cabling 
while  wearing  MOPP  IV  gear  was  considered  to  be  nearly  Impossible. 

°  The  operator  maintenance  manuals  were  considered  to  be  of  little 
value  and  were  little  used.  The  materials  provided  by  the 
contractor  were  considered  to  be  excellent. 

°  The  training  received  at  the  contractor  facility  was  considered  to 
be  inadequate.  Estimates  of  training  time  required  ranged  from 
four  to  eight  weeks  which  should  include  basic  computer  science. 

°  Typing  skills,  if  not  already  acquired,  should  also  be  trained. 

°  The  operators  felt  that  more  people  were  required  to  do  the  job 
with  the  AIM(6)  than  were  required  when  using  manual  methods  as  a 
manual  backup  was  needed  in  case  of  a  complete  system  malfunction. 

°  Several  operators  mentioned  that  a  separate  numeric  keyboard  would 
improve  their  work  efficiency. 

°  The  video  maps  provided  were  considered  to  be  of  virtually  no  use. 

The  primary  reason  cited  was  that  the  geographic  area  coverage  was 
too  small,  and  therefore,  a  hard  copy  map  covering  a  greater  area 
had  to  be  maintained. 
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°  The  tasks  considered  to  be  the  most  difficult  to  perform  while 
wearing  MOPP  IV  gear  were  cabling,  using  the  keyboard,  and 
camouflaging  the  vehicles. 

°  The  operators  felt  that  the  system  did  not  permit  use  of  their  MOS 
skills  to  the  fullest  advantage.  The  primary  reason  cited  was 
that  the  functions  available  on  the  system  were  lacking  in  many 
respects  and  different  from  many  they  normally  performed. 

°  No  major  health  or  safety  hazards  were  consistently  cited, 
although  several  of  the  TCAE  personnel  felt  that  camouflaging  the 
vehicles  while  wearing  MOPP  IV  gear  could  be  hazardous.  Loading 
and  unloading  the  MECAs  was  not  discussed  as  they  were  emplaced  in 
an  expandable  van  throughout  the  trials. 

°  Tracking  messages  requires  the  aid  of  the  system  supervisor  as  the 
operator's  message  number  and  the  VAX  message  number  are  not  the 
same.  This  results  in  lost  time  and  is  a  source  of  aggravation. 

°  Most  of  the  operators  felt  that  the  concept  behind  the  AIM  system 
was  good,  but  that  the  Implementation  left  something  to  be 
desired.  Lack  of  adequate  software  and  the  slow  speed  of  response 
were  considered  to  be  the  major  shortcomings. 

ft 

Discussion  of  AIM  operator  interviews.  The  fact  that  substitute 
equipment  and  only  a  very  preliminary  version  of  the  software  were  available 
for  the  trials  undoubtedly  gave  rise  to  many  of  the  negative  comments  received 
during  the  interviews.  It  Is  assumed  that  many  of  these  reported  shortcom¬ 
ings,  such  as  slow  system  functioning,  have  been  recognized  by  the  developer, 
and  therefore,  that  steps  will  be  taken  to  minimize  their  impact  in  the 
objective  equipment.  Nevertheless,  there  were  some  points  made  during  the 
interview  discussions  which  it  is  felt  need  to  be  considered  in  the  future 
hardware  and  software  development: 

One,  considerable  time  was  spent  in  reporting  problems  with  cables  and 
cabling.  In  addition  to  labels  being  lost  or  becoming  illegible,  operators 
reported  that  caps  came  off  the  cables  and  that  connectors  became  dirty. 
Threads  were  considered  to  be  too  fine  and  subject  to  stripping.  Operators 
also  felt  that  the  number  of  cables  was  excessive  and  could  be  reduced.  One 
group  suggested  that  the  Power  Load  Bank  be  permanently  mounted  on  the 
generator  trailer  and  hard  wired.  This  would  not  only  reduce  cabling  but 
would  eliminate  the  wait  for  the  Power  Load  Bank  to  cool  down  when  preparing 
for  movement. 

Two,  the  operators  felt  that  it  would  be  easy  to  accidentally  corrupt  the 
Order  of  Battle  (OB)  records.  An  attempt  to  prevent  this  was  made  by  having 
only  one  operator  change  OB  records  and  only  with  approval  by  the  System 
Supervisor.  Nevertheless,  the  operators  felt  that  in  the  heat  of  battle  the 
files  could  get  corrupted,  and  thought  that  a  less  easily  corrupted  system  was 
needed.  It  was  also  pointed  out  that  each  page  of  an  OB  record  must  be 
deleted  separately.  Failure  to  eliminate  all  pages  would  result  in  corruption 
of  the  OB  file. 


Three,  lack  of  adequate  geographical  area  coverage  was  the  main  reason 
cited  for  not  using  the  maps  available  on  the  monitors  at  the  MICROFIX 
stations.  However,  clutter  was  also  cited  as  a  reason  during  the  interviews. 
Overlap  of  symbols  made  it  very  difficult  to  determine  what  symbols  were 
actually  present  at  some  locations.  Smaller  symbols  or  some  form  of  "see 
through"  symbols  should  be  considered. 

Four,  concurrency  problems  were  a  headache  for  system  supervisors. 
Software  to  aid  the  supervisor  in  resolving  competing  requests  should  be 
developed . 

Five,  operators  reported  that  many  functions  available  at  the  MICROFIX 
stations  were  not  used  and  that  many  needed  functions  were  not  available.  Due 
to  the  MANPRINT  staff  member's  limited  familiarity  with  the  AIM(6)  and  the 
jobs  the  analysts  were  expected  to  perform,  specifics  were  not  requested. 
Software  developers  need  to  survey  analysts  to  determine  exactly  what 
functions  are  needed  and  ensure  they  are  included  in  later  releases. 

Personnel  Supervisor/Observer  Questionnaire  Findings 

A  total  of  10  questionnaires  were  returned,  seven  from  the  522d  MI  Bn  and 
three  from  2d  Armored  Division  G-2  personnel.  The  small  number  of  responses 
precludes  any  comparisons  between  the  'groups.  The  responses  to  the  items  are 
presented  in  Tables  8  and  9. 

°  Table  8  presents  the  responses  of  all  10  personnel  to  the  question 
"As  a  supervisor/observer,  did  the  system  seem  to  provide  support 
to  the  following  areas?"  A  summary  of  the  major  themes  in  the 
comment  sections  is  provided  below  the  table  and  is  keyed  to  the 
Roman  numerals  at  the  left  of  the  table. 
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Table  8 


Ratings  of  Support  for  System  Operations 


No 

support 

Inade¬ 

quate 

support 

Ade¬ 

quate 

support 

Good 

support 

Very 

good 

support 

Meets 

all 

needs 

No 

response 

I. 

DTOC  AIM  to  DTOCSE 

0 

3 

1 

2 

0 

0 

4 

II. 

CEWI  AIM  to  Ml/Bn 

TCAE 

0 

3 

4 

2 

1 

0 

0 

III. 

FSIC  to  Intelligence/ 
EU  Communications 

0 

1 

2 

6 

1 

0 

0 

IV. 

AIM(6)  FSIC  System  as 
a  Whole  to  Division 
I/EW  Operations 

0 

4 

5 

1 

0 

0 

0 

Note.  These  comments  were  received  from  the  10  supervisor/observers: 
Transmission  was  unreliable. 

System  provided  rapid  communications  when  working  properly. 

The  system  reliability  was  inadequate. 

The  system  was  not  user  friendly. 

°  Respondents  were  asked  to  indicate  the  major  strengths  and 

weaknesses  of  the  system  from  alternatives  provided.  They  were 
encouraged  to  check  as  many  of  the  alternatives  as  they  felt 
applied.  Space  for  comments  was  provided  where  additional 
strengths  and  weaknesses  could  be  listed  or  where  explanatory 
statements  concerning  alternatives  checked  could  be  made.  Table  9 
presents  the  data  for  the  "problem  areas"  and  Table  10  presents 
the  data  for  the  "best  points.”  Below  each  table  are  brief  state¬ 
ments  representing  the  major  themes  found  in  the  comments 
sections. 


Table  9 


Responses  to  the  Question  "What  are  the  Major  Problem  Areas  of  the  System?" 


Problem  Area  Number  of 

Responses 

Complexity  6 

Lack  of  software  to  support  functions  5 

Poor  hardware  design  1 

Poor  software  design  3 


Note  These  comments  were  received  from  the  10  supervisor /observers: 
The  software  doesn't  meet  the  operators'  needs. 

Software  is  too  inflexible. 

Procedures  for  composing  messages  are  too  cumbersome. 

Power  distribution  system  is  poor. 


Table  10 

Responses  to  the  Question  "What  are  the  Best  Points  About  the  System? 


Best  Points  Number  of 

Responses 

Communications  ability  6 

Ability  to  deal  with  large  volumes  of  data  6 

Speed  of  response  3 

Ease  of  operator  interface  0 


Note.  The  comment  was  received:  Disseminates  information  quickly. 

Discussion  of  Supervisor/Observer  Findings.  It  is  obvious  from  Table  8 
that  the  supervisors/observers  as  a  group  gave  the  system  rather  poor  ratings 
except  for  its  communications  capability.  This  sentiment  was  echoed  in  the 
question  on  the  system's  good  points  (Table  10)  as  well  as  in  the  written 
comments.  However,  only  one  rater  attributed  fault  to  the  hardware,  with  most 
stating  that  software  was  the  major  source  of  system  problems  (Table  9). 
Complexity  was  considered  a  problem,  but  this  may  have  been  due  to  software  as 
only  three  respondents  felt  the  system  placed  too  large  a  memory  burden  on  the 
operators.  Two  of  these  stated  that  it  was  the  FS1C  operators  who  were 
overburdened.  (AIM  6  operators  did  not  agree.  Eleven  of  26  responding 
operators  felt  the  system  placed  too  large  a  memory  burden  on  the  operator.) 


Many  of  Che  comments  received  were  comparable  co  those  obtained  from 
operators.  The  cabling  system  was  strongly  derided  by  one  respondent,  who 
added  that  the  labeling  could  not  be  read  in  the  dark  and  suggested  color 
coding  for  daylight  operations  and  “brailleing*1  for  night  operations.  Quick 
disconnects  were  also  suggested. 

The  inability  to  enter  and  leave  programs  at  any  point  was  criticized  as 
was  Inconsistency  between  comparable  inputs  among  programs  (e.g.,  date-time 
group).  Generation  of  queries  was  seen  as  too  complicated  and  message  formats 
were  seen  as  being  unwieldy. 


Physical  Measurements 

Selected  physical  measurements  were  made  inside  of  the  FS1C  shelter  to 
determine  compliance  with  standards  set  forth  in  OTEA  Test  Operations 
Procedure  1-1-059  (Soldier-Computer  Interface  Test  Procedures,  30  November 
1985).  A  list  of  the  measurements  obtained  is  shown  in  Appendix  I. 

°  The  diameter  of  the  keys  on  the  GRIDSE-T  keyboard,  the 
spacing  of  the  keys  on  the  keyboard,  and  the  size  of  the 
characters  on  the  GRIDSE-T  display  were  all  within  the 
standard. 

°  The  height  of  the  home  row  of  keys  on  the  keyboard  was  825mm 
above  the  floor  which  is  somewhat  above  the  720mm  +  25mm 
given  in  the  standard. 

°  The  width  and  height  of  the  legroom  available  were  within  the 
standards,  but  the  depth  of  approximately  400mm  was  below  the 
460mm  minimum  required. 

°  Kick  space  was  adequate  as  it  was  somewhat  above  the  100mm 
minimum  given  in  the  standard. 

°  The  seat  when  adjusted  to  a  minimum  height  measured  440mm, 
which  is  out  of  compliance  with  the  minimum  of  380mm  given  in 
the  standard.  The  maximum  height  adjustment  of  552mm  more 
than  met  the  standard  of  535mm. 

°  The  workspace  for  writing  measured  8.5  x  13  inches  which  is 
well  below  the  required  16  x  24  inches. 

°  The  noise  level  measured  at  the  approximate  location  of  the 
operator's  head  was  75.5db(A)  with  all  equipment  functioning 
except  the  printers,  and  77db(A)  with  the  printers  operating. 

These  measurements  are  well  within  the  safe  exposure  limits 
for  steady-state  noise  but  slightly  above  the  contractor's 
target  of  75db(A)  maximum.  (The  operators  were  questioned 
about  the  effects  of  noise  during  the  interviews.  None  of 
them  felt  that  the  noise  was  excessive  nor  that  it  interfered 

with  their  effectiveness,  although  four  reported  they  I 
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experienced  excessive  noise  in  responding  to  the  question¬ 
naire.  However,  since  the  operators  normally  wore  headsets, 
the  actual  noise  reaching  the  ears  was  considerably  attenuated 
from  that  measured  in  the  shelter. 

Two  of  the  measurements  which  were  not  in  compliance  with  the  TOP 
(legroom  depth  and  writing  workspace)  were  reported  as  being  deficient  by  the 
operators.  Two  others  (keyboard  height  and  minimum  seat  height)  were  not. 

Six  of  the  eight  operators  were  5’ 11"  or  greater  in  height,  so  minimum  seat 
height  and  actual  height  of  the  keyboard  would  not  be  expected  to  cause  a 
problem.  The  two  shortest  personnel  were  5' 7"  and  5'8"  in  height.  However, 
neither  of  them  reported  that  the  vertical  seat  adjustment  was  inadequate. 

CONCLUSIONS 

Virtually  everything  about  the  "field  trials"  of  the  ASAS  was  limited. 
Only  one  of  the  equipment  modules,  the  Forward  Sensor  Interface  and  Control 
(FSIC)  represented  the  objective  system  (the  system  to  be  developed).  The 
other  modules  were  substitutes  whose  similarities  and  dissimilarities  of 
operation,  compared  to  the  objective  system,  will  not  be  known  until  the 
objective  system  can  be  fielded  and  operated.  System  software,  which  strongly 
defines  the  capabilities  of  the  objective  system,  was  present  only  in  a  first 
version  that  substantially  lacked  features  intended  for  later  software 
versions. 

Insufficient  lead  time  was  available  to  enable  the  human  factors 
(MANPRINT)  investigator  to  receive  the  SCO  clearance  necessary  to  give  access 
to  the  system  during  actual  operation.  Data  collection  was  therefore  further 
limited.  Nevertheless,  as  a  result  of  these  field  trials,  it  is  probable  that 
ASAS  system  developers  incrementally  advanced  their  knowledge  and  readiness  to 
proceed  with  ASAS  system  development. 
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APPENDIX  A 

FSIC  OPERATOR  QUESTIONNAIRE 


The  responses  recorded  in  the  first  column  for  each  item  are  those  of  the 
forward  FSIC  operators.  Those  in  the  second  column  are  those  of  the  CEWI  FSIC 
operators. 


A-I 


(This  page  intentionally  left  blank.) 
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FSIC  OPERATOR 


QUESTIONNAIRE 


INFORMATION  PRIVACY  ACT  STATEMENT 


1.  Authority:  5  USC  301,  10  USC  3012,  Authority  for  the  Secretary  of 
the  Army  to  Issue  AR's;  44  USC  3101,  Authority  for  Collecting 
Necessary  Data* 

2.  Principal  Purpose:  To  collect  data  to  evaluate  the  human  fac¬ 
tors/manpower/  personnel/health  hazard/and  safety  training  aspects 
of  the  All  Source  Analysis  System  (ASAS)  during  operational  testing. 

3.  Routine  Uses:  The  data  collected  with  this  form  are  to  be  used  for 
research  purposes  only.  They  will  not  become  a  part  of  any 
Individual's  record  and  will  not  be  used  in  whole  or  in  part  in 
making  any  determination  about  an  individual. 

The  identifiers  (name  or  Social  Security  Number)  are  to  be  used  for 
administrative  and  statistical  control  purposes  only.  Full 
confidentiality  of  responses  will  be  maintained  in  the  processing  of 
these  data. 

4.  Mandatory  or  Voluntary  Disclosure  and  Effect  on  Individual  Not 
Providing  Information  -  Your  participation  in  this  research  is 
strictly  voluntary.  Individuals  are  encouraged  to  provide  complete 
and  accurate  Information  in  the  interests  of  the  research,  but  there 
will  be  no  effect  on  the  individuals  not  providing  all  or  any  part 
of  the  information. 

This  notice  may  be  detached  from  the  rest  of  this  form  and  retained 
by  the  individual  answering  the  questionnaire  if  so  desired. 
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FSIC  OPERATOR  QUESTIONNAIRE 


Name: 

Rank: 


Approximate  height: 
Approximate  weight: 

Age  _ years 

Time  in  service: 
Current  HOS: 


ft 


SSN: 

inches 


lbs 


months 


years 


Time  in  current  MOS: 


years 


Have  you  worked  on  TCAC?  _ Yes  _ No 

If  "Yes,”  in  what  position  (job  title)  _ 

If  "Yes,”  for  how  long?  _ years 


months 


months 


months 


Highest 

a. 

b. 

c. 

d. 

e. 

f . 

g* 

h. 


level  of  civilian  education  (Circle  letter  which  applies  to  you): 
No  high  school  diploma 
GED 

High  school  graduate 
1-2  years  of  college 
3-4  years  of  college 

College  graduate  (list  degree  and  major):  _ 

Advanced  degree  (list  degree  and  major):  _ 

Graduate  or  trade  school  (list  speciality):  _ 


Do  you  own  a  personal  computer?  _ Yes  _ No 

What  were  your  favorite  subjects  in  high  school? 


If  you  received  any  civilian  training  beyond  high  school,  what  was(were)  the 
primary  area(s)? 


Do  you  plan  on  making  the  Army  a  career?  _ Yes  _ No 

Regardless  of  your  current  career  intentions,  what  would  be  your  civilian 
occupational  choice(s)?  Please  be  realistic. 


INSTRUCTIONS 


This  questionnaire  contains  multiple-choice  items  asking  you  to  respond  on  a 
point  scale  or  to  chose  one  of  several  alternative  answers.  Read  each  item 
carefully  and  mark  the  one  rating  value  or  answer  that  most  closely  agrees  with 
your  personal  opinion  on  the  subject  being  considered. 

The  questionnaire  also  contains  items  in  which  you  are  asked  to  provide  written 
information.  Provide  all  the  information  you  think  appropriate. 

If  you  have  any  questions  concerning  the  meaning  of  any  items,  please  ask  the 
monitor  for  clarification. 
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Rate  your  pretest  understanding  of  the  tasks  required  of  you: 


0  1  Very  good 

1  1  Good 

21  In  between 

0  1  Poor 

0  0  Very  poor 

10  NR  (No  Response) 

Rate  your  posttest  understanding  of  the  tasks  required  of  you: 

0  1  Very  good 

1  3  Good 

10  In  between 

0  0  Poor 

1  0  Very  poor 

10  NR 


In  general,  how  easy-difficult  was  it  to  select  a  site  for  the  FISC 
vehicle  so  that  it  could  be  leveled? 


1  0  Very  easy 

3  3  Easy 

0  0  In  between 

0  0  Difficult 

0  0  Very  difficult 

0  1  NR 

Once  the  FSIC  vehicle  was  in  place  on  the  selected  site,  how  easy-dif¬ 
ficult  was  it  to  level  it  within  limits? 

1  1  Very  easy 

3  2  Easy 

00  In  between 
0  0  Difficult 

0  0  Very  difficult 

0  1  NR 


Do  you  feel  you  should  have  been  given  instruction  on  the  operation  of 
any  of  the  following  devices  at  the  beginning  of  training? 


Ye  8  No 


RT-524  radio  0  1 
KY-57  0  1 
HYX-57  3  1 
RT-1283a  radio  3  1 
K684A  4  1 
TD  1289  3  2 
C-6533  3  2 


Yes  No 

4  3 

4  3 

1  3 

1  3 

0  3 

1  2 

1  2 


Note .  NR  indicates  either  no  response  or  a  statement  that  the  item  was 
not  applicable  to  the  respondent. 
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6a.  How  easy-difficult  and  fast-slow  was  it  to  establish  and  maintain  Sync 
when  setting  up  communications? 

1  2  Easy  and  quick 

2  1  Easy  but  time  consuming 

0  0  Difficult  but  quick 

0  0  Difficult  and  time  consuming 

1  1  Other  (explain) 


6b.  On  the  average,  during  a  12-hour  shift,  how  often  did  you  lose  sync  (any 
or  all  lines)  and  have  to  reestablish? 

0  0  0 

1  1  1-2  times 

1  2  3-4  times 

2  1  5-6  times 

00  7  or  more  times 

7.  What  percentage  of  the  voice  messages  incoming  on  the  H-161  headset  were 
interfered  with  by  receipt  of  another  voice  message  on  the  other  channel? 

_ % 

forward  range:  0-10%  CEWI  range:  1-30% 

8.  How  easy-difficult  was  it  to  understand  voice  radio  communications? 

0  1  Very  easy 

3  2  Easy 

11  In  between 

0  0  Difficult 

0  0  Very  difficult 

9.  Could  you  read  the  output  of  the  Journal  printer  without  moving  and/or 
lifting  the  paper? 

2  3  Yes 

2  1  No 

If  "No,"  did  you  have  to  stand  up  to  read  it? 

1  0  Yes 

2  1  No 

(one  "yes"  respondent  completed  this  item) 
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10.  Could  you  read  the  output  of  the  Audit  printer  without  moving  and/or 
lifting  the  paper? 

1  1  Yes 

3  3  No 

If  "No,”  did  you  have  to  stand  up  to  read  It? 

2  2  Yes 

11  No 

11  NR 

11.  Were  you  able  to  hear  the  alarm  on  the  DCT  when  critical  or  flash 
messages  were  received? 

0  1  Practically  always 

0  0  Usually 

0  0  Half  the  time 

1  1  Seldom 

2  1  Practically  never 

11  NR 

12.  Were  the  tools  supplied  adequate  to  meet  your  needs  for  set-up,  tear- 
down,  and  conducting  PMCS? 

4  2  Yes 

0  1  No 

0  1  NR 

If  "No,"  please  explain: 


13a. 


Did  you  have  any  problems  remembering  how  to  make  interconnections  when 
the  equipment  first  arrived  at  Fort  Hood? 


13b. 


2  2  Yes 

2  2  No 

J 

Are  interconnections  adequately  coded  so  that  you  would  have  no  difficulty  making 
connections  after  a  layoff  of  60  days? 


3  1  Yes 
13  No 

If  "No,"  please  explain: 


M 


14.  Bow  easy-difficult  to  read  were  the  labels  on  the  equipment  such  as 
hazard  indicators,  dials,  switch  positions,  etc.? 

2  0  Very  easy 

1  2  Easy 

12  In  between 
0  0  Difficult 

0  0  Very  difficult 

15.  Please  list,  if  any,  those  labels,  dials,  markings,  etc.,  that  you  had 
difficulty  reading: 


16.  How  adequate-inadequate  was  the  lighting  for  each  of  the  following? 


Making  red  patch  panel 
connections 


Very 

adequate 


Inade-  Very 

Adequate  In  between  quate  Inadequate 


0  1 


11  12 


2  0  0  0 


Viewing  the  GRIDSE-T 

keyboard  2  2 

Reading  the  GRIDSE-T 

Display  (was  light  level 

too  high?)  2  2 

Setting  the  Crypto  devices  1  1 

Reading  printouts  from  the 
Journal  printer  2  2 

Reading  printouts  from  the 
Audit  printer  2  1 

Sanitizing  messages  1  1 


2  2  0  0 

2  2  0  0 

2  3  10 

2  2  0  0 

2  2  0  1 

3  3  0  0 


0  0  0  0 

0  0  0  0 
0  0  0  0 

0  0  0  0 

0  0  0  0 
0  0  0  0 


Reading  labels, 
meters,  etc. 

Tuning  TD  1289 


dials , 

1  1 
0  1 


0  3  3  0 

2  3  10 


0  0  0  0 
10  0  0 
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17.  How  adequate-inadequate  was  the  workspace  in  the  FSIC  shelter  for  each  of 
the  following  factors? 

Very 

Very  Ade-  In  Inade-  inade- 


adequate 

quate 

between 

quate 

quate 

NR 

Leg  room 

0 

0 

2 

2 

1 

1 

1 

1 

0 

0 

0 

0 

Elbow  room 

1 

0 

0 

2 

3 

2 

0 

0 

0 

0 

0 

0 

Seating 

Horizontal  adjustment 

1 

0 

3 

3 

0 

0 

0 

0 

0 

0 

0 

1 

Vertical  adjustment 

1 

0 

3 

3 

0 

0 

0 

0 

0 

0 

0 

1 

Backrest 

1 

0 

3 

2 

0 

0 

0 

1 

0 

0 

0 

1 

Cushioning 

1 

0 

3 

2 

0 

1 

0 

0 

0 

0 

0 

1 

Storage  room  ( for  papers 
personal  items,  etc.) 

1 

0 

0 

0 

0 

1 

0 

1 

3 

2 

0 

0 

Other  (specify) 


18.  How  frequently  did  you  experience  each  of  the  following? 

Frequency 


Cuts  0  2 
Electrical  shock  0  0 
Pinched/mashed  finger s/hands  2  3 
Excessive  noise  2  2 
Other  0  1 


If  you  checked  any  of  the  above,  please  explain  how: 

Other  _ 

Other  _ 

If  you  checked  any  of  the  above,  please  explain  how: 


19.  How  many  of  the  following  symptoms  did  you  have  while  operating  the  FSIC, 
and  how  long  after  your  shift  began  did  they  appear? 


Yes 


No 


How  long  on 
shift  (hours)* 


Eyestrain  0044 
Blurred  vision  0044 
Headaches  1133 
Dizziness  1034 
Fatigue  2123 
Neck  and  shoulder  stiffness  1133 
General  muscle  stiffness  0143 
Finger  fatigue  1034 
Backache  2123 
Hearing  difficulties  0044 


20.  How  easy-difficult  was  it  to  change  ribbons  in  the  printers? 


2  3  Did  not  change  a  ribbon 

0  1  Very  easy 

1  0  Easy 

10  In  between 
0  0  Difficult 

0  0  Very  difficult 

21.  How  easy-difficult  was  it  to  install  paper  and  adjust  the  paper  drive  in 
the  printers? 


0  0  Did  not  do 

0  0  Very  easy 

1  2  Easy 

31  In  between 
0  1  Difficult 

0  0  Very  difficult 


If  you  installed  paper,  how  many  times  did  you  do  this?  _ 

Time  estimates  varied  from  less  than  one  hour  to  eight  hours* 
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22.  How  easy-difficult  was  it  to  remove,  install  and  check  magnetic  tapes  in 
the  MDR-300? 

1  0  Did  not  do 

2  0  Very  easy 

0  2  Easy 

02  In  between 
0  0  Difficult 

1  0  Very  difficult 

If  you  removed,  installed,  checked  tapes,  how  many  times  did  you  do  this? 


23.  How  easy-difficult  was  it  to  load/key  the  crypto  devices? 


0  0  Did  not  do 

1  0  Very  easy 

0  1  Easy 

22  In  between 
1  1  Difficult 

0  0  Very  difficult 

24.  In  initializing  the  system,  how  adequate-inadequate  were  the  system 
prompts  for  each  of  the  following  operations? 


Booting  the  system 


Adequate  Inadequate  NR 

4  3  0  1 


CSF  initialization  (cold  start) 


4  3  0  1 


CSP  initialization  (warm  start)  3  3 

Resetting  date  and  time  4  4 

Configuring  message  file  disks  3  3 


1  1 
0  0 
1  1 


Terminal  sign  on 

Downloading  GRIDSET  2  3 
Entering  office  &  operator  names  1  2 
Entering  password  2  2 


2  1 

1  1  2  1 
1  1  11 


Activating  communication  lines 


3  3  11 


Initializing  DCC 
Loading  DMT 


3  3  11 

3  3  11 
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25.  Did  you  have  any  problems  shutting  down  the  system  and  preparing  for 
movement? 

Yes 

Sending  closing  message(s)  1  0 

Signing  off  the  terminal  1  0 

Zeroing  crypto  equipment  1  0 

Connecting  radio  for  convoy  ops  1  0 

Other,  explain: 


26.  Approximately  how  many  times  did  you  attempt  to  recall  inactive  messages? 

1  0  Never 

2  1  1-5  times 

0  0  6-10  times 

0  0  11-15  times 

1  3  More  than  15  times 

27.  Approximately  how  many  times  did  you  use  each  of  the  following  Recall 
modes?* 

3  4  MLN 

0  0  CDN 

2  1  ORI 

1  0  DTG 

0  1  TOT 

0  0  DAN 

0  0  OLD 

0  1  ABO 

Note.  Numbers  represent  the  number  of  operators  who  used  each  of  the  recall 
modes. 

28.  What  was  the  approximate  average  time  you  required  to  recall  the  desired 
message? 

1  0  Did  not  recall 

1  3  Less  than  1  minute 

0  1  1-2  minutes 

0  0  2-3  minutes 

0  0  3-4  minutes 

2  0  More  than  4  minutes 


29.  Did  you  ever  fail  to  obtain  any  message  you  tried  to  recall? 


1  0  Did  not  recall  messages 

2  2  Yes 

12  No 

30.  Overall,  how  easy-difficult  was  it  to  recall  messages? 

0  0  Very  easy 

1  3  Easy 

21  In  between 

0  0  Difficult 

0  0  Very  difficult 

10  NR 

31.  Did  you  ever  attempt  to  recover  messages  from  a  History  tape  after  a 

system  failure? 

0  0  Yes 

4  4  No 

If  "Yes,"  were  you  successful? 

_ _  Yes 

No 

Comment : 


32.  How  many  times  did  you  attempt  to  recall  messages  from  the  History  tape? 

3  4  Zero 

0  0  Once 

0  0  Twice 

0  0  Three  times 

1  0  Four  or  more  times 

33.  Overall,  how  would  you  rate  the  ease-difficulty  with  which  messages  could 
be  Altrouted? 

2  1  Very  easy 

2  3  Easy 

00  In  between 

0  0  Difficult 

0  0  Very  difficult 
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34.  Check  appropriate  box  for  the  adequacy  of  each  of  the  Altroutes: 


Altrouting  a  subset  Did  not 

use 


Worked  Did  not  work 
adequately  adequately  NR 


by  Precedence  code  4 
by  classification  4 
by  routing  Indicator  0 
SVC  to  SVP  2 


2 

2 

2 

1 


0 

0 

4 

2 


1 

1 

1 

3 


0  1 
0  1 
0  1 


35. 


How  well-poorly,  overall,  did 


the  Altroute  function  work? 


1  0  Very  well 

3  4  Well 

j  0  0  In-between 

0  0  Poorly 

0  0  Very  poorly 

36.  In  general,  how  easy-difficult  was  it  to  perform  each  of  the  following 
tasks? 

I 

T\J  J _ A-  T  r _  n  •  f  <•  i  «•  I  .  /■ 


Did  not 

Very 

In 

Diffi- 

Very  dif- 

perform 

easy  Easy 

between 

cult 

ficult  NR 

Message  creation 


Free  text  1 

0 

0 

1 

1 

2 

2 

1 

0 

0 

0 

0 

0 

0 

MI SMART  1 

1 

0 

0 

1 

2 

1 

1 

0 

1 

0 

0 

0 

0 

COMCAT  1 

1 

0 

0 

1 

2 

1 

1 

0 

0 

0 

0 

1 

0 

Message  editing  0 

0 

0 

1 

2 

3 

1 

0 

0 

0 

1 

0 

0 

0 

Message  disseminationO 

1 

0 

0 

2 

1 

1 

1 

1 

1 

0 

0 

0 

0 

Message  sanitization  0 

1 

1 

0 

3 

1 

0 

2 

0 

0 

0 

0 

0 

0 

Transcribing  messages 

Voice  to  text  1 

1 

0 

0 

0 

1 

3 

1 

0 

1 

0 

0 

0 

0 

Text  to  voice  0 

1 

0 

0 

1 

3 

3 

0 

0 

0 

0 

0 

0 

0 

Message  format 

Error  correction  0 

0 

0 

0 

2 

1 

1 

0 

0 

3 

1 

0 

0 

0 

Monitoring  system 

status  0 

0 

3 

0 

1 

2 

0 

1 

0 

1 

0 

0 

0 

0 

Adding  pilots  1 

0 

0 

0 

1 

2 

1 

1 

0 

1 

0 

0 

1 

0 

Creating  history  tapes 

(CRT  command)  2 

0 

0 

0 

1 

2 

0 

2 

1 

0 

0 

0 

0 

0 

Recovery  from  a 
corrupted  message 
file 


2  1 


0  0  1  1  0  0  0  1 


0  1 


1  0 


How  many 

times 

would  you 

estimate  that 

you  used  each 

of  the  following 

commands? 

Did 

1-2 

3-5 

More  than 

not  use 

times 

times 

5  times 

NR 

ARUPTD 

2 

3 

0 

1 

0 

0 

2  0 

0 

0 

CHG 

4 

4 

0 

0 

0 

0 

0  0 

0 

0 

COM 

1 

1 

0 

0 

0 

0 

3  3 

0 

0 

CRG 

4 

4 

0 

0 

0 

0 

0  0 

0 

0 

CROFCT 

2 

4 

0 

0 

0 

0 

1  0 

1 

0 

DCC 

1 

1 

0 

0 

0 

0 

3  3 

0 

0 

INTGEN 

4 

3 

0 

0 

0 

0 

0  0 

0 

1 

LML 

0 

0 

0 

0 

2 

0 

2  4 

0 

0 

LOGGEN 

4 

4 

0 

0 

0 

0 

0  0 

0 

0 

PLS 

4 

4 

0 

0 

0 

0 

0  0 

0 

0 

PRT 

4 

4 

0 

0 

0 

0 

0  0 

0 

0 

MSGRCL 

4 

1 

0 

1 

0 

1 

0  1 

0 

0 

RTC 

4 

4 

0 

0 

0 

0 

0  0 

0 

0 

SHS 

3 

3 

0 

0 

0 

0 

0  0 

1 

1 

SQS 

2 

0 

0 

0 

1 

0 

1  4 

0 

0 

STS 

4 

3 

0 

0 

0 

0 

0  1 

0 

0 

SUS 

0 

1 

0 

0 

0 

0 

4  3 

0 

0 

SWO 

4 

4 

0 

0 

0 

0 

0  0 

0 

0 

OPR 

1 

3 

0 

0 

1 

0 

2  1 

0 

0 

UPT 

1 

3 

0 

0 

0 

0 

3  1 

0 

0 

38.  Did  you  build  an  automatic  routing  data  base  file  (ARMDB _ )  to 

reduce  your  workload  in  sending  specific  types  of  messages  to  requestors? 

0  0  Yes 

3  0  No 

14  NR 

39.  How  easy-difficult  was  it  to  understand  the  rules  for  determining 
addresses  for  ACTION  and  INFO  messages? 

0  0  Very  easy 

0  0  Easy 

21  In  between 

0  1  Difficult 

I  1  Very  difficult 

II  NR 
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40.  Approximately  what  percentage  of  your  operating  time  would  you  estimate 
that  you  spent  seeking  help  (from  contractors  personnel,  the  HLP  menu, 
manuals  and  class  notes,  other  operators,  etc)? 

0  1  102  or  less 

1  1  10-202 

1  1  20-302 

2  1  30-402 

0  0  Other  (estimate) 


41.  How  many  times  each  day,  on 
Help  (HLP)  command? 


the  average,  would  you  estimate  you  used  the 


14  0 

2  0  1-2 

1  0  3-4 

0  0  5-6 

0  0  Other  (estimate) 

42.  How  adequate-inadequate  was  the  HLP  menu  in  informing  you  how  to  enter 
the  commands? 

1  3  Did  not  use 

0  0  Very  adequate 

1  0  Adequate 

20  In  between 

0  1  Inadequate 

0  0  Very  inadequate 

43.  Did  you  use  checklists  in  initializing  the  equipment  when  the  Pilot 
Studies  started  on  November  17? 


44.  Do  you  still  use  any  checklists  in  performing  your  job? 


If  "Yes,"  what  do  you  use  checklists  for? 
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45.  If  you  received  a  message  (error  message,  fatal  message,  informational 
message,  or  unusual  occurrence  message)  that  you  didn't  understand,  what 
was  the  approximate  average  time  required  to  find  out  what  the  message 
meant  and  decide  what  action,  if  any,  you  should  take? 

0  0  Did  not  receive  any  of  these  messages 

0  1  Did  not  receive  any  messages  I  didn't  understand 

00  1  minute  or  less 

0  0  1-2  minutes 

0  0  2-3  minutes 

1  1  3-5  minutes 


3  2  More  than  5  minutes, 

please  estimate 

minutes 

46. 

What  percentage  of  the  time 

would  you  estimate 

you 

had  to  seek  help 

to 

understand: 

Approximate 

If 

received 

#  messages* 

% 

estimate 

Error  messages 

4  1 

50- 

-99 

"  70 

Fatal  messages 

1  1 

NR 

100 

Information  messages 

1  0 

30 

- 

Unusual  occurrence  messages 

1  1 

50 

100 

Note.  Entries  are  number  of  operators  who  received  a  message 

of  that 

type. 

47. 

Did  you  ever  generate  a: 

Yes  No 

NR 

Queue  Statistics  Report? 

1 

3  2 

1 

1 

0 

Station  Status  Report? 

2 

3  1 

1 

1 

1 

System  History  Report? 

1 

2  2 

2 

1 

0 

PLA  Statistics  Report? 

1 

1  2 

3 

1 

0 

48. 

How  useful  to  you  were  each 

of 

the  following  displays? 

Very 

Moderately 

Of 

Of 

Did 

useful 

Useful  useful 

little  use 

no 

use 

not 

see 

CSP  dynamic  status 

display  0  2 

0 

1  1  0 

0 

0 

0 

1 

3 

0 

Station  status 

report  0  0 

1 

1  1  0 

0 

1 

1 

2 

1 

0 

Queue  statistics 

report  2  0 

2 

10  0 

0 

1 

0 

2 

0 

0 

System  history 

report  0  0 

0 

1  1  0 

0 

1 

0 

2 

3 

0 

Very  Moderately  Of  Of  Did 

useful  Useful  useful  little  use  no  use  not  see 

Line  logging 

report  10010001  0230 

Plain  language 

statistics  report  1  0000001  0231 

CSP  system 
configuration 

report(s)  10001001  1211 

49.  How  easy-difficult  to  read  were  the  displays  on  the  GR1DSE-T? 

3  0  Very  easy 

1  4  Easy 

00  In  between 

0  0  Difficult 

0  0  Very  difficult 

50.  How  easy-difficult  to  read  are  the  numerals,  letters  and/or  commands  on 
the  GRIDSE-T  keyboard? 

2  0  Very  easy  Note.  The  two  keyboards  differed.  The  commands 

2  1  Easy  on  the  CEWI  FSIC  keyboard  were  pasted  over  the 

00  In  between  keys*  and  with  use,  the  print  rubbed  off  and 

0  1  Difficult  became  difficult  to  read. 

0  2  Very  difficult 

51.  Does  the  GRIDSE-T  keyboard  provide  an  adequate  typing  "feel?" 

4  1  Yes 

0  0  No 

52.  How  easy-difficult  would  it  be  to  use  the  GRIDSE-T  wearing  MOPP  IV  gear? 

0  0  Very  easy 

0  0  Easy 

00  In  between 

1  1  Difficult 

3  2  Very  difficult 

0  1  NR 

53.  Does  the  system  provide  adequate  prompts  to  prevent  operator  errors? 

0  0  Nearly  always 

2  2  Most  of  the  time 

1  0  Many  additional  prompts  are  needed 

02  A  few  additional  prompts  are  needed 

1  0  Some  additional  prompts  are  needed 
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54.  Does  the  system  provide  adequate  prompts  to  correct  operator  errors? 

0  1  Nearly  always 

2  1  Most  of  the  time 

0  2  Many  additional  prompts  are  needed 

10  A  few  additional  prompts  are  needed 

1  0  Some  additional  prompts  are  needed 

55.  How  adequately-inadequately  did  the  FARM  function  work  for  intercepting 
the  particular  messages  you  wished  to  see? 

0  0  Very  adequately 

0  1  Adequately 

30  In  between 

1  1  Inadequately 

0  0  Very  inadequately 

0  2  NR 

56.  Do  you  feel  that  operating  the  FSIC  places  too  large  a  memory  burden  on 
the  operators? 

0  1  Yes 

4  3  No 

57.  During  peak  workload  times  did  your  performance  of  any  tasks  become  less 
effective? 

3  2  Yes 

12  No 

58.  Did  your  performance  of  any  task  become  unsatisfactory  during  peak 
workload  times? 

3  2  Yes 

12  No 

59.  Are  there  any  tasks  or  functions  during  which  it  would  be  possible  to 
accidentally  erase  or  modify  the  system's  files? 

0  2  Yes 

2  2  No 

2  0  NR 

60.  Do  you  feel  that  system  resnonse  time  to  any  commands  was  excessive? 

1  2  Yes 

3  2  No 

If  "Yes,"  list  those  commands  below: 


61.  Are  there  any  functions  or  tasks  that  you  feel  are  more  difficult  to 
accomplish  with  the  FSIC  than  with  procedures  you  employed  before? 


Please  list  any  such  functions  or  tasks: 


62.  Compared  to  times  when  the  FSIC  was  fully  operational,  how  easy-dif f icult 
was  it  to  keep  up  with  your  work  during  degraded  operations  (e.g.,  during 
a  communications  failure)? 

0  0  Very  easy 

0  0  Easy 

32  In  between 

1  1  Difficult 

0  1  Very  difficult 

63.  How  easy-difficult  do  you  feel  it  would  be  for  a  sophisticated  enemy 
force  to  locate  the  FSIC  by  electronic  means? 

4  2  Very  easy 

0  1  Easy 

00  In  between 

0  0  Difficult 

0  0  Very  difficult 

0  1  NR 

64.  How  easy-difficult  do  you  feel  it  would  be  for  a  sophisticated  enemy 
force  to  locate  the  FSIC  by  acoustic  means? 

1  1  Very  easy 

1  0  Easy 

22  In  between 

0  0  Difficult 

0  0  Very  difficult 

0  1  NR 


The  FSIC  is  designed  to  provide  an  automated  and  manual  interface  between 
sensors  and  the  MI  battalion  TCAE.  To  help  us  evaluate  how  well  this  worked 
please  assess  the  FSIC's  performance  in  the  following  areas: 

65.  Providing  data  communications  between  sensors  and  the  TCAE,  and  convert¬ 
ing  messages  into  common  formats. 

0  0  FSIC  failed  to  perform  these  tasks 

1  1  Performance  was  inadequate 

1  2  Performance  was  adequate,  but  the  FSIC  has  major  problems 

2  0  Performance  was  good,  but  there  are  still  some  problems 

0  1  Performance  was  very  good 

0  0  Performance  was  excellent 
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66.  Providing  a  capability  for  transcribing  voice  messages  into  automated 
message  formats. 

0  0  FSIC  failed  to  perform  these  tasks 

1  1  Performance  was  inadequate 

1  0  Performance  was  adequate,  but  the  FSIC  has  major  problems 

1  1  Performance  was  good,  but  there  are  still  some  problems 

1  1  Performance  was  very  good 

0  0  Performance  was  excellent 

0  1  NR 

67.  Providing  data  communications  between  the  Ml  Battalion  TCAE  and  the 
sensors: 

1  0  FSIC  failed  to  perform  these  tasks 

1  2  Performance  was  inadequate 

1  0  Performance  was  adequate,  but  the  FSIC  has  major  problems 

0  1  Performance  was  good,  but  there  are  still  some  problems 

1  1  Performance  was  very  good 

0  0  Performance  was  excellent 

68.  Assisting  in  the  identification,  sanitization  and  routing  of  combat 
information  to  the  Brigade. 

1  0  FSIC  failed  to  perform  these  tasks 

1  2  Performance  was  Inadequate 

1  0  Performance  was  adequate,  but  the  FSIC  has  major  problems 

1  0  Performance  was  good,  but  there  are  still  some  problems 

0  1  Performance  was  very  good 

0  0  Performance  was  excellent 

0  1  NR 

69.  Providing  Automated  Interface  with  the  AIM: 

0  0  FSIC  failed  to  perform  these  tasks 

0  1  Performance  was  inadequate 

0  1  Performance  was  adequate,  but  the  FSIC  has  major  problems 

2  1  Performance  was  good,  but  there  are  still  some  problems 

0  1  Performance  was  very  good 

0  0  Performance  was  excellent 

2  0  NR 

70.  What  are  the  major  problem  areas  of  the  system? 

3  3  Complexity 

0  1  Lack  of  software  to  support  functions 

0  2  Poor  hardware  design 

0  2  Poor  software  design 

1  2  Other  (specify) 


I 
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71.  What  are  the  best  points  about  the  system? 


3  1  Communications  capability 

2  2  Ability  to  deal  with  large  volume^  of  data 

1  1  Speed  of  response 
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appendix  b 

AIM  OPERATOR  QUESTIONNAIRE 

The  responses  recorded  in  the  first  column  for  each  item  are  those  of 
the  DTOC  AIM  operators.  Those  in  the  second  column  are  those  of  the 
CEWI  AIM  operators.  Those  in  the  third  column  are  the  combined 
responses  of  all  AIM  personnel  including  the  supervisors.  Exceptions 
to  this  scheme  of  recording  are  noted  on  the  questionnaire  for 
specific  items. 
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AIM  OPERATOR/ INTELLIGENCE  ANALYST 


QUESTIONNAIRE 


INFORMATION  PRIVACY  ACT  STATEMENT 


1.  Authority:  5  USC  301,  10  USC  3012,  Authority  for  the  Secretary  of 
the  Army  to  Issue  AR's;  44  USC  3101,  Authority  for  Collecting 
Necessary  Data. 

2.  Principal  Purpose:  To  collect  data  to  evaluate  the  human  fac¬ 

tors/manpower/  personnel/health  hazard/and  safety  training  aspects 
of  the  All  Source  Analysis  System  (ASAS)  during  operational  testing. 

3.  Routine  Uses:  The  data  collected  with  this  form  are  to  be  used  for 
research  purposes  only.  They  will  not  become  a  part  of  any 
individual's  record  and  will  not  be  used  in  whole  or  in  part  in 
making  any  determination  about  an  individual. 

The  identifiers  (name  or  Social  Security  Number)  are  to  be  used  for 
administrative  and  statistical  control  purposes  only.  Full 
confidentiality  of  responses  will  be  maintained  in  the  processing  of 
these  data. 

4.  Mandatory  or  Voluntary  Disclosure  and  Effect  on  Individual  Not 
Providing  Information  -  Your  participation  in  this  reseach  is 
strictly  voluntary.  Individuals  are  encouraged  to  provide  complete 
and  accurate  information  in  the  interests  of  the  research,  but  there 
will  be  no  effect  on  the  individuals  not  providing  all  or  any  part 
of  the  information. 

This  notice  may  be  detached  from  the  rest  of  this  form  and  retained 
by  the  individual  answering  the  questionnaire  if  so  desired. 


I 
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AIM  OPERATOR/ INTELLIGENCE  ANALYST  QUESTIONNAIRE 

Name:  _ 

Rank: _  SSN: _ 

Approximate  height:  _ ft  _ inches 

Approximate  weight:  _ lbs 

Age  _ years  _ months 

Time  in  service:  _ years  _ months 

Current  MOS:  _ 

Time  in  current  MOS:  _ years  _ _ months 

Have  you  worked  on  TCAC?  _ Yes  _ No 

If  "Yes,”  In  what  position  (job  title)  _ 

If  "Yes,"  for  how  long?  _ years  _ months 

What  is  your  position  (job  title)  for  the  current  ASAS  field  trials? 


Highest  level  of  civilian  education  (Circle  letter  which  applies  to  you): 

a.  No  high  school  diploma 

b.  GED 

c.  High  school  graduate 

d.  1-2  years  of  college 

e.  3-4  years  of  college 

f.  College  graduate  (list  degree  and  major):  _ 

g.  Advanced  degree  (list  degree  and  major):  _ 

h.  Graduate  or  trade  school  (list  speciality):  _ 

Do  you  own  a  personal  computer?  _ Yes  _ No 

What  were  your  favorite  subjects  in  high  school? 


If  you  received  any  civilian  training  beyond  high  school,  what  was(were)  the 
primary  area(s)? 


Do  you  plan  on  making  the  Army  a  career?  _ Yes  _ No 

Regardless  of  your  current  career  intentions,  what  would  be  your  civilian 
occupational  choice(s)?  Please  be  realistic. 
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INSTRUCTIONS 


This  questionnaire  contains  multiple-choice  items  asking  you  to  respond 
on  a  point  scale  or  to  chose  one  of  several  alternative  answers.  Read 
each  item  carefully  and  mark  the  one  rating  value  or  answer  that  most 
closely  agrees  with  your  personal  opinion  on  the  subject  being  con¬ 
sidered. 

The  questionnaire  also  contains  items  in  which  you  are  asked  to  provide 
written  information.  Provide  all  the  information  you  think  appropriate. 

If  you  have  any  questions  concerning  the  meaning  of  any  items,  please  ask 
the  monitor  for  clarification. 
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1.  Rate  your  understanding  of  the  tasks  required  of  you  at  the  completion  of 
training  at  Martin-Marietta: 

226  Very  good 

5  6  16  Good 

215  In  between 
101  Poor 
000  Very  poor 

2.  Rate  your  understanding  of  the  tasks  required  of  you  at  present: 

348  Very  good 

2  3  8  Good 

227  In  between 
304  Poor 
001  Very  poor 

3.  In  general,  how  easy-difficult  was  it  to  select  a  site  for  the  AIM 
shelter  vehicle  so  that  it  could  be  leveled? 

022  Very  easy 

3  18  Easy 

224  In  between 
226  Difficult 
000  Very  difficult 

3  2  8  NR 

4.  Once  the  AIM  vehicle  was  in  place  on  the  selected  site,  how  easy-dif¬ 
ficult  was  it  to  level  it  within  limits? 

012  Very  easy 

4  2  9  Easy 

214  In  between 
123  Difficult 
000  Very  difficult 

3  3  10  NR 

5.  Answer  items  5a  through  5d  based  on  your  experiences  at  Martin-Marietta: 

5a.  How  easy-diff icult  was  it  to  level  the  vehicle  so  that  movement  of  the 
MECAs  during  on/offloading  could  be  easily  controlled? 

239  Very  easy 
517  Easy 

114  In  between 
1  0  1  Difficult 

011  Very  difficult 
13  6  NR 


L. 
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5b.  In  offloading  MECAs,  how  easy-difficult  was  it  to  attach/detach  the  MLD 
from  the  MECAs? 

235  Very  easy 

6  1  13  Easy 

227  In  between 
000  Difficult 
000  Very  difficult 
0  3  3  NR 

5c.  How  easy— difficult  was  it  to  attach/detach  the  wheels  while  on/off loading 
the  MECAs? 

213  Very  easy 
6  4  15  Easy 

206  In  between 
022  Difficult 
000  Very  difficult 
0  2  2  NR 

5d«  How  easy-difficult  was  it  to  move  the  MECAs  to/from  the  operating 
positions  to/ from  the  vehicle? 

236  Very  easy 

6  2  12  Easy 

015  In  between 
202  Difficult 
011  Very  difficult 
0  2  2  NR 

6.  Have  you  been  injured  in  any  way  while  positioning  the  MECAs  (either  at 
Martin-Marietta  or  at  Fort  Hood)? 

115  Yes 
9  7  22  No 

0  11  NR 

(If  "Yes,"  please  explain:) 


ai'iii'i**  .t< .. 


7.  Were  all  netting  shelter  cables,  connectors,  etc.,  adequately  labeled  or 
coded  to  insure  proper  connections  without  damage  to  equipment  or  injury 
to  personnel? 

9  5  19  Yes 

0  4  8  No 

10  1  NR 

(If  "No,"  specify  which  connectors  and/or  cables  were  easily  confused:) 


8.  Did  you  know  when  you  were  to  "bump  start"  the  ECU  in  the  AIM  shelter  (if 
"off"  for  more  than  4  hours,  or,  if  exposed  to  temperatures  below  50°F)? 

112  Yes 

9  8  25  No 

0  0  1  NR 

9.  Did  you  follow  a  checklist  when  powering  up  the  AIM  shelter  components? 
204  Yes,  every  time 

426  Almost  every  time 

2  5  13  Just  for  the  first  trials  until  I  learned  the  procedures 

12  4  I  knew  the  procedures  well  before  the  test  started  and  didn't 

need  a  checklist 

10  1  NR 

10.  Was  lighting  in  the  shelter  adequate  for  powering  up  and  powering  down 
the  AIM  shelter  components? 

10  9  28  Yes 

0  0  0  No 

If  "No,"  please  explain  where  additional  lighting  is  needed: 


11.  How  easy  was  it  to  mount  the  magnetic  tape? 

002  Very  easy 
2  5  12  Easy 

001  In  between 
101  Difficult 
001  Very  difficult 
7  4  1 1  NR 
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Were  all  cables  and  connectors  adequately  labeled  or  coded  to  insure 
proper  connection  to  the  MECAs  without  damage  to  the  equipment  or  injury 
to  personnel? 


10  8  22  Yes 

0  16  No 

(If  "No,"  specify  which  connectors  and/or  cables  were  easily  confused:) 


13.  Did  you  have  any  problems  in  powering  up  your  workstation? 

Yes  No  NR 

Forgetting  proper  position  of 

MICROFIX  switches  00  10  8  01 

Energizing  CPU  and  disk 

drive  in  wrong  sequence  00  10  8  01 

Forgetting  to  unlock  head 

on  video  player  21  87  01 

Loading  and  adjusting  paper 

in  printer  13  95  01 

Setting  up  crypto  equipment  00  87  22 

Setting  up  AN/VRC  46  radio  10  77  22 

14.  Did  you  use  the  map  displays  on  your  CRT? 

517  Yes 
5  7  20  No 

0  11  NR 

If  "No,"  why  not? 


If  "Yes,"  answer  items  14a  through  14e : 
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14a.  How  easy-dif f icult  is  it  to  read  the  lettering  on  the  maps  presented  on 
your  CRT? 

112  Very  easy 

4  15  Easy 

328  Marginal 
101  Difficult 
Oil  Very  difficult 

I  4  11  NR 

14b.  How  adequate-inadequate  is  the  coverage  of  territory  on  the  maps 
presented  on  your  CRT? 

01  1  Very  adequate 

40  5  Adequate 

20  2  In  between 

23  5  Inadequate 

II  4  Very  inadequate 

14  11  NR 

14c.  How  easy-difficult  was  it  to  place  symbols  on  the  CRT  maps? 

000  Very  easy 
516  Easy 

113  In  between 

326  Difficult 
Oil  Very  difficult 
1  4  12  NR 

14d.  How  easy-difficult  was  it  to  move  or  delete  symbols  on  your  CRT  maps? 

000  Very  easy 
607  Easy 
023  In  between 
326  Difficult 
Oil  Very  difficult 
1  4  11  NR 

14e.  How  easy-difficult  was  it  to  distinguish  among  the  map  symbols  on  your 
CRT  maps? 

000  Very  easy 
213  Easy 
516  In  between 
124  Difficult 
113  Very  difficult 
1  4  12  NR 

15.  Supervisor  Only:  Overall,  how  easy-difficult  were  each  of  the  GESCAN 

functions  to  accomplish?  Note .  Operators  were  requested  verbally  to 
respond  if  they  had  performed  these  functions.  Only  one  CEWI  operator 
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responded  and  is  not  recorded.  The  first  column  presents  the  responses 
of  the  DTOC  operators,  and  the  second  includes  the  supervisors. 


Very 

easy 

Easy 

In 

between 

Diffi¬ 

cult 

Very 

diffi 

cult 

- 

NR  or 

Not 

used 

Creating  queries 

1 

2 

2 

4 

4 

8 

1 

3 

1 

1 

1 

1 

Editing  queries 

1 

2 

3 

5 

2 

7 

0 

0 

1 

2 

3 

3 

Choosing  correct  data 
bases  for  queries 

1 

4 

1 

4 

3 

6 

1 

1 

0 

0 

4 

4 

Choosing  correct  terms 
for  queries 

0 

0 

1 

2 

5 

10 

1 

4 

1 

1 

2 

2 

Deciding  where  additional 
terms  should  be  added  to 
queries  (top  &  bottom)  0 

1 

0 

1 

4 

8 

1 

3 

1 

1 

4 

5 

Determining  fields  for 
query  terms 

0 

2 

4 

4 

2 

4 

0 

3 

1 

3 

3 

3 

Deleting  terms  from 
queries 

0 

2 

5 

6 

1 

5 

0 

0 

0 

1 

4 

5 

Using  Logic  equations 
to  narrow  or  broaden 
searches 

0 

0 

3 

5 

1 

6 

2 

3 

0 

0 

4 

5 

Choosing  and  using 
proximity  and  distance 
qualifiers 

0 

0 

0 

0 

0 

1 

1 

2 

0 

0 

9 

16 

Aborting  queries 

0 

3 

3 

8 

0 

0 

1 

1 

0 

0 

6 

7 

Copying  queries 

1 

2 

3 

8 

0 

0 

1 

0 

0 

0 

5 

9 

Deleting  queries 

L 

4 

4 

8 

1 

2 

1 

1 

0 

0 

3 

4 

Monitoring  queries 

0 

3 

5 

7 

0 

2 

0 

1 

0 

0 

5 

6 

Printing  queries 

1 

2 

2 

6 

2 

2 

0 

0 

0 

0 

5 

9 

Renaming  queries 

1 

2 

3 

7 

0 

0 

2 

2 

0 

0 

4 

8 

Obtaining  query 
directory 

1 

3 

2 

5 

0 

0 

1 

2 

0 

0 

6 

9 

Running  queries 

1 

3 

4 

10 

0 

1 

1 

1 

0 

0 

4 

4 
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(item  15  cont'd) 

Very 

easy 

Easy 

In 

between 

Diffl 

cult 

- 

Very 

diffi¬ 

cult 

NR  or 
Not 
used 

Changing  qualifiers 

for  existing  queries 

0 

1 

2 

4 

0 

2 

1 

3 

0 

0 

7 

9 

Obtaining  Hit  file  for 

browsing 

0 

4 

1 

5 

1 

1 

1 

1 

0 

0 

7 

8 

Obtaining  Hit  file 

directory 

0 

3 

1 

3 

0 

0 

1 

1 

0 

0 

8 

12 

Understanding  Hit  file 

directory 

0 

3 

0 

2 

1 

2 

1 

1 

0 

0 

8 

11 

Finding  specific  terms 

in  messages 

0 

3 

1 

3 

1 

2 

1 

2 

0 

1 

7 

8 

Editing  search  results 

0 

1 

1 

4 

0 

3 

1 

1 

0 

1 

8 

9 

Tagging  Hit  files 

0 

1 

1 

1 

0 

2 

1 

1 

0 

0 

8 

14 

Annotating  Hit  files 

0 

1 

1 

1 

0 

2 

1 

1 

0 

0 

8 

14 

Printing  messages 

2 

5 

3 

6 

2 

3 

0 

1 

0 

0 

3 

4 

Deleting  Hit  files 

2 

4 

3 

6 

1 

3 

0 

0 

0 

0 

4 

6 

Creating  thesaurus 

0 

2 

1 

1 

0 

2 

0 

2 

0 

0 

9 

12 

Using  directory  of 

thesauri 

0 

3 

1 

4 

0 

1 

0 

0 

0 

0 

9 

11 

Deleting  terms  in  a 

thesaurus 

0 

3 

1 

2 

0 

0 

0 

1 

0 

0 

9 

13 

Copying  a  thesaurus 

0 

1 

1 

3 

0 

0 

0 

0 

0 

0 

9 

15 

Renaming  a  thesaurus 

0 

1 

1 

3 

0 

0 

0 

0 

0 

0 

9 

15 

Changing  qualifiers  in 

a  thesaurus 

0 

1 

0 

1 

1 

2 

0 

0 

0 

0 

9 

15 

Printing  a  thesaurus 

0 

0 

1 

3 

0 

0 

0 

0 

0 

0 

9 

16 

Using  a  thesaurus  in 

a  query 

0 

1 

1 

4 

0 

1 

0 

0 

0 

0 

9 

13 

Deleting  a  thesaurus 

0 

3 

1 

3 

0 

0 

0 

0 

0 

0 

9 

13 
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16.  MICROFIX  Workstation  Operators  Only:  How  easy-difficult  was  It  to 
perform  each  of  the  terminal  control  processor  operations?  Note . 
Supervisors  did  not  use  MICROFIX  equipment. 


Easy 

In  between 

Difficult 

NR 

Not 

or 

Used 

Log  on  to  MICROFIX 
&  establish  V2  link 

9 

9 

0 

1 

0 

0 

0 

0 

Display  summary  of 
queued  messages 

10 

8 

0 

1 

0 

0 

0 

0 

Display  a  specific 
message 

9 

9 

1 

0 

0 

0 

0 

0 

Execute  options  on  a 
message 

9 

9 

1 

0 

0 

0 

0 

0 

Transfer  a  file  to  V2 
for  printing 

5 

6 

4 

3 

1 

0 

0 

0 

Upload  a  message  to  V2 
for  transmission 

7 

6 

1 

3 

1 

0 

1 

0 

Review  temporary 
message  queue 

Accessing  temporary 
queue 

9 

4 

3 

0 

0 

0 

1 

2 

Selecting  message 

9 

3 

0 

4 

0 

0 

0 

3 

Downloading  to  MICROFIX 
data  base  8 

1 

2 

4 

0 

1 

0 

3 

Return  control  to 
terminal 

9 

9 

0 

0 

0 

0 

1 

0 

17a.  How  would  you  rate  the  response  time  to  commands  at  your  station? 

000  Very  fast 
112  Fast 
428  In  between 
5  3  12  Slow 

035  Very  slow 
0  0  1  NR 


I 


B-13 


17b.  Did  you  notice  any  differences  in  system  response  time  by: 


NR  Yes  No 


Time  of  day? 

0 

2 

4 

0 

0 

0 

10 

7 

24 

Number  of  stations  operating? 

0 

2 

3 

7 

1 

15 

3 

6 

10 

Intensity  of  the  situation? 

0 

2 

4 

6 

2 

13 

4 

5 

11 

Type  of  requirement? 

0 

2 

4 

5 

3 

15 

5 

4 

9 

18.  Is  the  organization  of  information  displayed  helpful  for  doing  your  job? 

4  4  10  Yes 

2  4  12  No 

4  16  NR 

19.  Are  there  enough  computer  commands  which  specify  various  functions  to 
accomplish  your  job? 

8  2  14  Yes 

15  9  No 

12  5  NR 

20.  Overall,  how  easy-difficult  was  it  to  generate  MISMART  messages? 

134  Very  easy 
6  4  12  Easy 

327  In  between 
002  Difficult 
001  Very  difficult 
0  0  2  NR 

21.  Were  there  any  tasks  more  difficult  or  time  consuming  to  accomplish  with 
the  analyst  stations  than  with  previous  methods  you  have  used?  If  so, 
please  list  those  functions: 


22.  Are  there  any  tasks  or  functions  during  which  it  would  be  possible  to 
accidentally  erase  or  modify  the  system's  files  without  adequate  warning? 

014  Yes 
7  6  14  No 

3  2  10  NR 
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23a.  When  you  received  a  response  to  tasking  during  normal  operations,  how 

would  you  rate  the  adequacy  of  response  time  to  your  tasking  messages? 

000  Very  satisfactory 
328  Satisfactory 
347  In  between 
416  Unsatisfactory 
023  Very  unsatisfactory 
0  0  4  NR 

23b.  When  you  received  a  response  to  tasking  during  peak  periods  of  opera¬ 
tions,  how  would  you  rate  the  adequacy  of  response  time  to  your  tasking 

messages? 

000  Very  satisfactory 
237  Satisfactory 
124  In  between 
719  Unsatisfactory 
034  Very  unsatif actory 
0  0  4  NR 

24.  Oo  you  feel  that  using  this  system  places  too  large  a  memory  burden  on 
the  operator? 

4  1  11  Yes 

6  8  15  No 

0  0  2  NR 

25.  How  easy-difficult  was  it  to  understand  voice  communications? 

101  Very  easy 
224  Easy 
135  In  between 
000  Difficult 
001  Very  difficult 
6  4  17  NR 

26.  How  safe  did  you  feel  in  accomplishing  each  of  the  following  operations? 
(Answer  based  on  your  experience  at  Martin-Marietta) 

NR  or 


Very  In  Very  Did 

safe  Safe  between  Unsafe  unsafe  not  do 


Unloading  the  MECAs 
from  the  vehicle 

1 

1 

2 

7 

2 

10 

1 

1 

5 

0 

2 

2 

0 

2 

5 

1 

1 

4 

Attaching  wheels  & 

handles  to  the  MECAs 

1 

4 

5 

7 

1 

12 

1 

0 

1 

4 

0 

4 

0 

0 

3 

1 

0 

3 

Moving  the  MECAs  into 
position  for  use 

4 

4 

8 

5 

3 

14 

1 

2 

5 

0 

0 

0 

0 

0 

0 

0 

0 

1 
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NR  or 

Very  In  Very  Did 

safe  Safe  between  Unsafe  unsafe  not  do 

Removing  wheels  & 
handles  from  the  MECAs 
when  in  position. 

Attaching  cables, 
connectors,  etc. 


1  4  7  7  2  13 

2  4  9  7  5  16 


1  2  4  0  1  2 

1  0  2  0  0  0 


0  0  0  1  0  2 

0  0  0  0  0  1 


Operating  an 
Analyst  Station 

Moving  the  MECAs  to 
the  transport  vehicle 

Loading  the  MECAs  on 
the  transport  vehicle 


5  6  14  5  3  10 

3  4  7  6  2  13 

1  2  3  6  1  8 


0  0  0  0  0  0 

0  2  4  0  0  0 

2  1  5  0  2  2 


0  0  0  0  0  4 

0  0  1  113 

0  2  5  1  1  5 


If  you  are  a  DT0C  AIM  system  operator  or  data  collector,  answer  questions 
27a  through  27g. 


27a.  Does  the  AIM  provide  a  limited  ADP  support  capability  to  the  following 
DTOC  functions? 

(NOTE:  CEWI  AIM  personnel  did  not  respond.  The  first  column  presents 
the  DTOC  operators'  responses  and  the  second  includes  supervisors' 
responses.) 


System  Management: 

0  0  None 

0  2  Inadequate  support 

5  7  Limited  support,  but  some  problem  areas 

3  4  Very  good  support 

0  0  Fully  meets  the  requirements  of  my  job 

2  2  NR 

27b.  Situation  development/assessment  (near  battle  analysts,  deep  battle 
analysts) : 


0  0 
1  4 
7  8 
1  1 
0  0 
1  2 


None 

Inadequate  support 

Limited  support,  but  some  problem  areas 
Very  good  support 

Fully  meets  the  requirements  of  my  job 


NR 
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27c.  Target  Development: 

0  1  None 

1  3  Inadequate  support 

4  6  Limited  support,  but  some  problem  areas 

1  1  Very  good  support 

0  0  Fully  meets  the  requirements  of  my  job 

4  4  NR 

27d.  Collection  Management: 

1  2  None 

3  5  Inadequate  support 

2  4  Limited  support,  but  some  problem  areas 

2  2  Very  good  support 

0  0  Fully  meets  the  requirements  of  my  job 

2  2  NR 

27e.  All  Source  Analysis: 

0  1  None 

1  2  Inadequate  support 

7  10  Limited  support,  but  some  problem  areas 

1  1  Very  good  support 

0  0  Fully  meets  the  requirements  of  my  job 

11  NR 

27f.  Storage,  Retrieval  and  Correlation  of  Data: 

0  0  None 

2  3  Inadequate  support 

5  7  Limited  support,  but  some  problem  areas 

3  4  Very  good  support 

0  1  Fully  meets  the  requirements  of  my  job 

27g.  Intelligence  Reporting: 

0  0  None 

3  6  Inadequate  support 

5  6  Limited  support,  but  some  problem  areas 

1  2  Very  good  support 

0  0  Fully  meets  the  requirements  of  my  job 

1  1  NR 

If  you  are  a  CEWI  AIM  system  operator  or  data  collector,  answer  questions 
28a  through  28f. 

(NOTE:  DTOC  personnel  did  not  respond.  The  first  column  shown  is  for 

CEWI  AIM  operators  and  the  second  column  includes  supervisors'  respon¬ 
ses.) 
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28.  Does  Che  AIM  provide  a  limited  ADP  support  capability  to  the  following 
TCAE  functions? 

28a.  System  Management: 

1  3  None 

1  2  Inadequate  support 

3  4  Limited  support,  but  some  problem  areas 

1  1  Very  good  support 

0  0  Fully  meets  the  requirements  of  my  job 

3  3  NR 

28b.  Mission  Management: 

1  3  None 

3  4  Inadequate  support 

3  4  Limited  support,  but  some  problem  areas 

0  0  Very  good  support 

0  0  Fully  meets  the  requirements  of  my  job 

2  2  NR 

28c.  Sigint  Analysis  (Comint  and/or  Elint): 

1  5  None 

2  2  Inadequate  support 

4  4  Limited  support,  but  some  problem  areas 

0  0  Very  good  support 

0  0  Fully  meets  the  requirements  of  my  job 

2  2  NR 

28d.  All  Source  Input/Analysis: 

0  3  None 

1  1  Inadequate  support 

5  5  Limited  support,  but  some  problem  areas 

1  1  Very  good  support 

0  0  Fully  meets  the  requirements  of  my  job 

2  3  NR 

28e.  Storage,  Retrieval  and  Correlation  of  Data: 

1  3  None 

3  5  Inadequate  support 

1  1  Limited  support,  but  some  problem  areas 

2  2  Very  good  support 

0  0  Fully  meets  the  requirements  of  my  job 

2  2  NR 
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28f.  Intelligence  Reporting: 


0  1  None 

0  2  Inadequate  support 

6  7  Limited  support,  but  some  problem  areas 

0  0  Very  good  support 

1  1  Fully  meets  the  requirements  of  my  job 

2  2  NR 

29.  How  many  of  the  following  symptoms  did  you  have  while  operating  an 
Analyst  Station,  and  how  long  after  your  shift  began  did  they  appear? 


NR 

Yes 

No 

How  long  on 
shift  (hours)* 

Eyestrain 

1 

0 

3 

6 

8 

17 

3 

1 

8 

Blurred  vision 

0 

0 

2 

2 

3 

6 

8 

6 

20 

Headaches 

0 

0 

2 

4 

5 

10 

6 

4 

15 

Dizziness 

0 

1 

3 

1 

1 

2 

9 

7 

23 

Fatigue 

0 

0 

2 

7 

6 

16 

3 

3 

10 

Neck  &  shoulder 
stiffness 

0 

0 

2 

8 

4 

18 

2 

5 

8 

General  muscle  stiffness 

0 

0 

2 

6 

4 

14 

4 

5 

12 

Finger  fatigue 

0 

0 

2 

1 

1 

3 

9 

8 

23 

Backache 

0 

0 

2 

6 

4 

15 

4 

5 

11 

Hearing  difficulties 

0 

0 

2 

0 

2 

2 

10 

7 

24 

*Response  varied  from  less 

than 

one  1 

hour  to 

12 

hours . 

30.  How  easy-difficult  would  it  be  to  prepare  the  AIM(6)  (including  Analyst 
Stations)  while  wearing  MOPP  IV  gear? 

000  Very  easy 
0  0  0  Easy 

113  In  between 
617  Difficult 
3  7  17  Very  difficult 

0  0  1  NR 


What  tasks  would  be  most  difficult: 


31.  How  easy-difficult  would  it  be  to  use  your  keyboard  while  wearing  MOPP  IV 
gear? 

000  Very  easy 
0  0  0  Easy 

416  In  between 
538  Difficult 
1  5  13  Very  difficult 

0  0  1  NR 

32.  What  are  the  major  problem  areas  of  the  system? 

4  2  10  Complexity 

5  3  14  Lack  of  software  to  support  functions 

5  1  11  Poor  hardware  design 

1  8  13  Poor  software  design 

328  Other  (specify)  _ 

33.  What  are  the  best  points  about  the  system? 

019  Communications  capability 
4  5  13  Ability  to  deal  with  large  volumes  of  data 

012  Speed  of  response 

225  Ease  of  operator  interface  with  the  system 

506  Other  (specify)*  _ 

♦Responses  largely  Indicated  that  the  system  concept  was  good. 
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APPENDIX  C 

FSIC  OPERATOR  INTERVIEW  QUESTIONS 


C-I 


(This  page  intentionally  left  blank.) 
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FSIC  OPERATORS 


Items  for  Structured  Interviews 


1.  Are  the  most  frequently  employed  controls  conveniently  located  and  grouped 
for  ease  of  performance?  What  should  be  changed? 

2.  Do  you  know  of  any  safety  or  health  hazards  that  were  not  covered  in 
adequate  detail  in  the  questionnaires? 

3.  What  would  you  most  like  to  see  changed  in  FSIC  operating  procedures? 

4.  Jid  you  have  any  problems  in  emplacing,  connecting,  operating,  or  march 
ordering  the  power  generation  equipment? 

5.  What  areas  or  what  tasks  do  you  think  should  be  particularly  emphasized 
in  future  training  for  FSIC  operators? 

6.  Do  you  have  any  suggestions  for  improving  the  usefulness  of  manuals  and 
other  printed  materials? 

7 .  What  skills  are  required  on  the  job  by  FSIC  operators  that  were  not 
Included  in  your  MOS  training  (or  used  in  selection)? 

8-  Does  the  FSIC  require  any  increases  (decreases)  in  personnel  to  accomplish 
the  equivalent  tasks  with  manual  methods? 

9.  What  personnel  attributes  (aptitudes  or  aptitude  levels,  physical 

requirements,  etc.)  are  required  for  FSIC  operations  that  are  not  required 
for  manual  operations? 

10.  What  tasks,  and  why,  would  be  most  difficult  to  accomplish  while  wearing 
MOPP  IV  gear? 

11.  How  would  you  assess  your  workload?  Did  it  ever  get  so  heavy  that  you 
got  significantly  behind  in  processing  messages? 

12.  Stress: 

(a)  Name  any  stressful  tasks,  procedures  or  functions  which  stress  the 
operator  sufficiently  to  reduce  effectiveness  to  unacceptable 
levels. 

(b)  For  each:  Does  the  degree  or  duration  of  the  stress  lead  to 
unsatisfactory  performance? 

Supervisors:  What  personal  characteristics  do  you  feel  are  required  for 
a  competent  FSIC  operator? 
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AIM  OPERATOR  INTERVIEW  QUESTIONS  - 


1 


! 


D1 


(This  page  intentionally  left  blank.) 


■ 


i 


D-2 


AIM  ANALYSTS 


Items  for  Structured  Interviews 


Did  your  queries  produce  the  information  you  expected  them  to  produce? 

Are  the  procedures  for  creating  queries  flexible  enough  for  you  to  be 
able  to  request  the  information  you  need? 

Are  the  procedures  for  creating  queries  complete  enough  for  you  to  be 
able  to  request  the  information  you  need? 

Were  the  data  bases  generally  adequate  to  provide  the  desired  data? 

What  problems  did  you  have  in  creating  queries? 

Is  it  possible  to  track  queries,  that  is,  to  determine  the  status  of 
given  queries? 

Are  the  most  frequently  employed  controls  conveniently  located  and 
grouped  for  ease  of  performance?  What  should  be  changed? 

Do  you  know  of  any  safety  or  health  hazards  that  were  not  covered  in 
adequate  detail  in  the  questionnaires? 

Did  you  have  any  problems  in  emplacing,  connecting,  operating  or  march 
ordering  the  power  generation  equipment? 

What  areas  or  what  tasks  do  you  think  should  be  particularly  emphasized 
in  future  training  for  analysts  operating  computer-based  intelligence 
systems? 

Do  you  have  any  suggestions  for  improving  the  usefulness  of  manuals  and 
other  printed  materials? 

What  skills  are  required  on  the  job  that  were  not  included  In  your  MOS 
training  (or  used  in  selection)? 

Does  the  AIM(6)  system  as  configured  require  any  increases  (decreases)  in 
personnel  to  accomplish  the  same  functions  that  were  previously  performed 
by  manual  methods? 

What  personnel  attributes/aptitudes  (aptitude  levels,  physical  require¬ 
ments,  etc.)  are  required  for  AIM(6)  operations  that  are  not  required  for 
manual  operations? 

What  tasks,  and  why,  would  be  most  difficult  to  accomplish  while  wearing 
MOPP  IV  gear? 

How  would  you  assess  your  workload?  Did  it  ever  get  so  heavy  that  you 
got  significantly  behind  in  processing  your  work? 


6. 


Stress: 


(a)  Name  any  tasks,  procedures,  or  functions  which  stress  the  analyst 
sufficiently  to  reduce  effectiveness  to  unacceptable  levels? 


(b)  For  each:  Does  the  degree  or  duration  of  the  stress  lead  to  unsatis¬ 
factory  performance? 


Supervisors:  What  personal  characteristics  do  you  feel  are  required  for  a 
competent  analyst  on  a  computer-based  intelligence  system? 
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APPENDIX  E 

PERSONNEL  SUPERVISOR/OBSERVER  QUESTIONNAIRE 

Responses  in  the  first  column  are  those  of  CEWI  personnel  and  those  __ 

in  the  second  column  are  those  of  DTOC  personnel. 
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PERSONNEL  SUPERVISOR/ OBSERVER 


QUESTIONNAIRE 


INFORMATION  PRIVACY  ACT  STATEMENT 


1.  Authority:  5  USC  301,  10  USC  3012,  Authority  for  the  Secretary  of 
the  Army  to  Issue  AR's;  44  USC  3101,  Authority  for  Collecting 
Necessary  Data. 

2.  Principal  Purpose:  To  collect  data  to  evaluate  the  human  fac¬ 
tors/manpower/  personnel/health  hazard/and  safety  training  aspects 
of  the  All  Source  Analysis  System  (ASAS)  during  operational  testing. 

3.  Routine  Uses:  The  data  collected  with  this  form  are  to  be  used  for 
research  purposes  only.  They  will  not  become  a  part  of  any 
individual's  record  and  will  not  be  used  in  whole  or  in  part  in 
making  any  determination  about  an  individual. 

The  identifiers  (name  or  Social  Security  Number)  are  to  be  used  for 
administrative  purposes  only.  Full  confidentiality  of  responses 
will  be  maintained  in  the  processing  of  these  data. 

4.  Mandatory  or  Voluntary  Disclosure  and  Effect  on  Individual  Not 
Providing  Information  -  Your  participation  in  this  research  is 
strictly  voluntary.  Individuals  are  encouraged  to  provide  complete 
and  accurate  information  in  the  interests  of  the  research,  but  there 
will  be  no  effect  on  the  individuals  not  providing  all  or  any  part 
of  the  information. 

This  notice  may  be  detached  from  the  rest  of  this  form  and  retained 
by  the  individual  answering  the  questionnaire  if  so  desired. 
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PERSONNEL  SUPERVISOR  AND  OBSERVER  QUESTIONNAIRE 


Name : 

Rank: 

SSN: 

Time  in  service: 

years 

months 

Current  MOS: 

Time  in  current  MOS: 

years 

months 

What  is  your  position  (job  title)  for  the  ASAS  field  trials? 


Have  you  worked  on  TCAC?  _ Yes  _ No 

If  "Yes,"  in  what  position  (job  title)  _ 

If  "Yes,"  for  how  long?  _ years  _ months 

Highest  level  of  civilian  education  (Circle  letter  which  applies  to  you): 

a.  High  school  graduate 

b.  1-2  years  of  college 

c.  3-4  years  of  college 

d.  College  graduate  (list  degree  and  major):  _ 

e.  Advanced  degree  (list  degree  and  major): 

f.  Graduate  or  trade  school  (list  speciality):  _ 


Do  you  own  a  personal  computer?  _ Yes  _ No 

What  military  schools  or  other  training  have  you  completed  (e.g.,  officer 
basic,  officer  advanced,  Sergeants  Major  Academy,  etc.,  specify  branch  where 
appropriate) ? 


Do  you  plan  on  making  the  Army  a  career?  _ Yes  _ No 


Regardless  of  your  current  career  intentions,  what  would  be  your  civilian 
occupational  choice(s)? 


As  a  Supervisor/Observer ,  did  the  system  seem  to  provide  support  to  the 
following  areas? 


1.  DTOC  AIM  to  DTOCSE  Operations: 

00  a.  No  support 

21  b.  Inadequate  support 

10  c.  Adequate  support,  but  major  problems  still  exist 

02  d.  Good  support,  but  some  problems  still  exist 

GO  e.  Very  good  support 

00  f.  Meets  all  needs  for  this  function 

40  NR  (No  Response) 

Comment : 


2.  CEWI  AIM  to  MI  Battalion/TCAE  Operations: 

00  a.  No  support 

30  b.  Inadequate  support 

22  c.  Adequate  support,  but  major  problems  still  exist 

11  d.  Good  support,  but  some  problems  still  exist 

10  e.  Very  good  support 

00  f.  Meets  all  needs  for  this  function 

Comment : 


3.  FISC  to  Intelligence/EW  Communications: 

00  a.  No  support 

01  b.  Inadequate  support 

20  c.  Adequate  support,  but  major  problems  still  exist 

42  d.  Good  support,  but  some  problems  still  exist 

10  e.  Very  good  support 

00  f.  Meets  all  needs  for  this  function 


Comment 


AIM(6)/FSIC  System  as  a  Whole  to  Division  I/EW  Operations: 


0  0 
4  0 
3  2 
0  1 
0  0 
0  0 


a.  No  support 

b.  Inadequate  support 

c.  Adequate  support,  but  major  problems  still  exist 

d.  Good  support,  but  some  problems  still  exist 

e.  Very  good  support 

f.  Meets  all  needs  for  this  function 


Comment : 


What  are  the  major  problem  areas  of  the  system?  (Check  as  many  as 
appropriate;  put  an  asterisk  (*)  by  the  one  you  feel  is  the  greatest 
problem. ) 


6  2 
5  1 
1  2 
3  2 
1  0 


a.  Complexity 

b.  Lack  of  software  to  support  functions 

c.  Poor  hardware  design 

d.  Poor  software  design 

e.  Others  (specify)  _ 


Comment : 


What  are  the  best  points  about  the  system?  (Check  as  many  as  appropria¬ 
te;  put  an  asterisk  (*)  by  the  one  you  feel  is  the  best  attribute  of  the 
system. ) 

62  a.  Communications  ability 

62  b.  Ability  to  deal  with  large  volumes  of  data 

32  c.  Speed  of  response 

00  d.  Ease  of  operator  interface  with  the  system 

01  e.  Others  (specify)  _ 


Comment : 
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Do  you  feel  the  system  as  is  places  too  great  a  memory  burden  on  the 
operators? 


Yes  _  No 

3  0  3  3 

10  NR  (No  Response) 


If  "YES,"  which  one(s)? 

I 


8.  Please  list  any  tasks,  subtasks,  displays,  procedures,  or  functions  which 
you  feel  must  be  changed  for  the  system  to  be  acceptable  to  you  (e.g., 
creating  queries,  generating/editing  messages,  map  displays,  recalling 
messages,  error  correction  procedures,  etc.): 


APPENDIX  F 

FSIC  OPERATOR  PERFORMANCE/ATTITUDE/KNOWLEGE 
RATINGS  FORM 
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FSIC  OPERATOR  PERFORMANCE  RATINGS 


Rater: 


Please  rate  each  of  the  FSIC  operators  on  each  of  the  items  below  using  the 
following  scale: 


Very  satisfactory  =  5 
Satisfactory  =  4 
In  between  -  3 
Unsatisfactory  =  2 
Very  unsatisfactory  =  1 


Operator 

ll|2|3|4|5|6|7|8| 


Computer  literacy 


MOS  knowledge 

Overall  knowledge  of  FSIC 
operations 


Flexibility  in  using  alternative 
means  of  accomplishing  tasks 
with  the  FSIC 

Ability  to  rapidly  sanitize 
messages 


Attitude  towards  job 


Adaptability  to  unexpected 
or  emergency  situations 


Ability  to  work  under  stress 


Ability  to  work  with  others 


Overall  performance  during  test(s) 


Note .  Forms  given  to  the  raters  had  the  names  of  the  personnel  rated  at  the 
tops  of  the  columns. 


APPENDIX  G 

AIM  OPERATOR  PERFORMANCE/ATTITUDE/KNOWLEDGE 
RATING  FORM 
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AIM  (6)  OPERATOR  PERFORMANCE  RATINGS 


Rater: 


Please  rate  each  of  the  AIM  operators  on  each  of  the  items  below  using  the 
following  scale: 


Very  satisfactory  *  5 
Satisfactory  *  4 
In  between  ■  3 
Unsatisfactory  ■  2 
Very  unsatisfactory  -  1 


Operator 

|1|2|3|4|5|6|7|8| 


Computer  literacy 
MOS  knowledge 

Overall  knowledge  of  AIM(6) 
operations 

Flexibility  in  using  alternative 
approaches  of  accomplishing  tasks 
with  the  MICROFIX 


Attitude  towards  job  I _ i 

Ability  to  rapidly  sanitize  I  ! 

messages  I _ I 

Adaptability  to  unexpected  i  I 

or  emergency  situations  I _ I 

Ability  to  work  under  stress  I _ I 

Ability  to  work  with  others  I _ I 

Overall  performance  during  test(s)  I _ I 


Note.  Forms  given  to  the  raters  had  the  names  of  the  personnel  to  be  rated  at 
the  tops  of  the  columns. 


G-3 


APPENDIX  H 


MEMORANDUM  TO  ASAS  TEST  PERSONNEL  AND  DATA  COLLECTORS 


m*  .  r  r.  T.: 


(This  page  intentionally  left  blank.) 


Memorandum  to:  ASAS  Test  Personnel  and  Data  Collectors 


From:  MANPRINT  Staff 

Subject:  MANPRINT  Issues  In  ASAS. 

1.  Manpower  and  Personnel  Integration  (MANPRINT)  Is  a  new  Initiative  which 
is  described  in  AR  602-2.  MANPRINT  is  an  umbrella  term  encompassing  human  factors 
engineering,  manpower,  personnel,  training,  health  hazard  assessment  and  system 
safety.  The  MANPRINT  initiative  was  developed  because  the  US  Congress  wanted  to 
know  “Are  the  Armed  Forces  getting  what  they  are  paying  for  in  total  weapon  system 
performance  or  are  critical  resources  being  wasted  to  acquire  high  capability 
technology  that  exceeds  the  bounds  of  human  performance?”  AR  602-2  describes  the 
Army's  approach  to  ensuring  that  the  human  user  will  be  given  full  consideration  in 
all  phases  of  system  acquisition  in  the  future.  The  Commanding  General,  OTEA,  is 
assigned  the  responsibility  to: 

a.  Ensure  MANPRINT  Issues  are  identified,  tested,  and  evaluated 
during  testing  and  evaluation  for  which  OTEA  is  responsible. 

b.  Ensure  pertinent  MANPRINT  data  are  collected  during  user  testing 
for  which  OTEA  is  responsible  and  are  made  available  for  use  by  other  MANPRINT 
activities  and  for  continuous  comprehensive  evaluation. 

c.  Verify  that  soldiers  used  in  testing  are  representative  of  the 
user  population. 

2.  Unfortunately,  none  of  the  MANPRINT  staff  has  an  SCI  clearance. 
Therefore,  it  will  not  be  possible  for  them  to  observe  the  testing.  Normally, 
hypotheses  concerning  potential  problems  in  MANPRINT  areas  are  developed  during 
test  observation  and  clarified  through  the  use  of  questionnaires  and  posttest 
interviews.  Since  direct  observation  by  staff  members  will  not  be  possible  during 
the  current  ASAS  testing,  the  staff  is  asking  you  to  help  provide  this  information. 
It  is  realized  that  you  will  be  kept  busy  with  other  duties.  However,  you  may 
notice  such  things  as  signs  of  extreme  frustration  during  certain  activities, 
errors  made  in  performing  specific  tasks,  behavior  which  might  damage  the  equipment 
or  lead  to  personnel  injury,  or  near  accidents.  If  you  do,  make  a  mental  note  and 
report  the  incident  to  CPT  Slater.  Please  report  any  incident  you  think  might  be 
relevant.  Over-reporting  rather  than  under-reporting  is  desired.  By  combining 
reports  from  all  sources,  we  should  be  able  to  find  the  problems  which  need 
correcting. 


3.  As  a  further  aid  to  you,  brief  definitions  of  the  six  MANPRINT  domains 
are  attached. 


MANPRINT  DOMAINS* 


Human  Factors  Engineering 

A  comprehensive  technical  effort  to  integrate  into  Army  doctrine,  materiel 
development  and  materiel  acquisition  (to  insure  operational  effectiveness)  all 
relevant  information  concerning: 

Human  characteristics 
Skill  capabilities 
Performance 
Anthropometric  data 
Bio-medical  factors 
Safety  factors 
Training 

Manning  implications 

Human  Factors  Engineering  is  concerned  with  the  design,  development,  testing, 
evaluation  and  deployment  of  manned  systems  so  that  soldiers  would  be  able  to 
operate  and  maintain  military  systems  at  their  optimum  performance  levels. 

This  includes  the  systematic  investigation  of  how  the  design  of  the  soldier's 
job  and  the  tools  that  are  provided  affect  his/her  capacity  to  do  the  job. 

The  major  emphasis  is  on  system  reliability  and  performance,  soldier-system 
compatibility,  and  understanding  of  cost-benefits  and  ultimately,  the  achieve¬ 
ment  of  user  acceptance. 

Manpower 

Manpower  is  the  personnel  strength  expressed  in  terms  of  the  number  of  men  and 
women  available  to,  or  required  by  the  Army. 

Manpower  is  concerned  not  merely  with  the  accumulation  of  space  requirements 
but  also  with  (1)  distribution  of  quantity,  (2)  distribution  of  gender,  (3) 
distribution  of  skills  and  experience,  (4)  consideration  of  costs  and  bill 
payers,  (5)  force  structure  changes,  and  (5)  workload  and  grade  structure. 

Personnel 

The  characteristics  of  an  individual  soldier  (as  opposed  to  a  manpower  space). 
Personnel  includes  consideration  of  MOS,  specialty  code,  grade,  and  aptitudes. 

The  following  statement  by  General  Creighton  Abrams  puts  the  personnel  domain 
in  the  proper  perspective: 

"By  people,  I  do  not  mean  'personnel.'  I  do  not  mean  end  strength.  I  do  not 
mean  'percent  fill'  or  any  of  those  labels  which  refer  to  people  as  commodity. 
1  mean  living,  breathing,  serving  human  beings.  They  have  needs  and  interests 
and  desires.  They  have  spirit,  and  will,  and  strengths  and  abilities.  They 
have  weaknesses  and  faults.  They  have  NAMES." 

*Adapted  from  MANPRINT  PRIMER  (Draft),  US  Army  Deputy  Chief  for  Personnel,  April 
1986. 
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Training 

Imparting  the  requisite  knowledge,  skills  and  abilities  to  qualify  Army 
personnel  for  use,  operation,  maintenance  and  support  of  Army  systems  or 
equipment . 

In  the  most  basic  terms,  training  is  the  process  which  prepares  soldiers  to 
perform  jobs.  The  soldier  is  given  a  series  of  tasks,  aptly  named  "soldier 
tasks,"  which  describe  what  the  Army  wants  the  soldier  to  do.  Performance 
standards  are  established  to  measure  how  well  the  Army  wants  the  soldier  to 
do  the  task;  and  finally,  performance  reflects  the  soldier's  ability  to 
accomplish  the  desired  tasks. 


Health  Hazard  Assessment 

An  existing  or  likely  condition  Inherent  to  the  operation  or  use  of 
materiel  that  can  cause  death,  injury,  acute  or  chronic  illness,  dis¬ 
ability,  and/or  reduced  job  performance  of  personnel  by  exposure  to: 

Shock/Recoil 

j  Vibration 

Noise  (including  steady  state,  impulse  and  blast  overpressure) 

Humidity 
Toxic  gases 
Toxic  chemicals 

Ionizing  and  non-ionizing  radiation  (including  x-rays,  gamma 
rays,  magnetic  fields,  microwaves,  radio  waves,  and  high 
Intensity  light) 

Lasers 

Heat  and  cold 
Oxygen  deficiency 
Blunt/ sharp  trauma 
Pathogenic  micro-organisms 

I 

System  Safety 

The  optimum  degree  of  safety  and  health  features,  within  the  bounds  of 
operational  effectiveness,  time  and  cost,  attained  by  using  system  safety 
engineering  and  management  principles  to  identify  hazards  and  reduce  risks 
throughout  a  system's  life  cycle. 
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LIST  OF  PHYSICAL  MEASUREMENTS  OBTAINED  FOR  THE  FSIC 
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Physical  Measurements 


Height  of  keyboard  (720mm  +25mm) 

Leg  room  (640mm  H  x  510nm  W,  x  460nm  D) 

Kickspace  (100mm  minimum) 

Adjustment  range  of  seating  (380-535mm) 

Workspace  for  writing  (24  In  x  16  in) 

Diameter  of  keys  on  keyboard  (19-38mm) 

Spacing  of  keys  on  keyboard  (6mm) 

Size  of  characters  on  GRIDSET  (16  minutes  of  visual  angle  at  16  inches) 

Size  of  characters  on  Analyst  Displays  (16  minutes  of  visual  angle  at  normal 
viewing  distance) 

Noise  levels  both  with  and  without  the  printers  operating. 
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Requirements  from  OTEA  Database 
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DATA  REQUIREMENTS 


1.  All  questionnaire  and  interview  data  will  be  manually  scored,  reduced  and 

interpreted. 

2.  Data  relevant  to  human  factors  required  from  data  bases  (FSIC). 

a.  List  of  and  description  of  circumstances  surrounding  any  injury  to  personnel. 

b.  List  of  any  hardware  or  software  damages  due  to  operator  error. 

c.  List  of  times  the  number  of  messages  in  or  sent  to  a  queue  exceeded  95%  of 
the  capacity  of  the  queue  (including  the  DCT  buffers). 

d.  Time  required  to  sanitize  messages  (M,  a). 

e.  Number  of  messages  awaiting  sanitization  during  normal  workload  (M  and 
Maximum). 

f.  Times  between  receipt  of  SCI  messages  at  Forward  FSIC  and  the  beginning  of 
transmission  of  sanitized  message  (M,  0). 

g.  Number  of  errors  made  in  completing  formatted  messages,  by  message  type 
(N,  %). 

h.  Times  to  raise  each  antenna  from  start  of  cranking  to  lock  (M,  o ,  Range).  This 
could  be  added  to  the  form  on  p  F-21  of  Data  Management  Plan. 

i.  List  of  functions  being  performed  each  time  the  software  crashed  or  hung. 

j.  Times  required  to  tune  the  radios  and  load  the  crypto  device  (M,  a). 

k.  Times  to  prepare  freetext  messages  with  the  gridset  (M,  o). 

l.  Times  to  prepare  freetext  messages  with  the  DCT  (M,  o). 

m.  Times  required  to  become  operational  after  the  generator  is  detached  and  the 
CUCV  stopped  (M,  a ). 

n.  Times  required  to  start  road  movement  after  the  initial  order  to  move  is  given 
(M,  Range). 

o.  By  shift,  number  of  operator  prepared  (or  edited)  messages  of  each  type  sent 
by  each  FSIC: 

1)  DCT 

2)  Voice  to  Brigade  or  Battalion 

3)  Voice  to  sensor 

4)  Freetext  messages  prepared  on  GRIDSET 

5)  COMCAT  formatted  messages  prepared  on  GRIDSET 

p.  By  shift,  by  FSIC,  the  number  of  times  a  HELP  menu  was  requested. 
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Data  relevant  to  human  factors  required  from  data  bases  (AIM/6). 

a.  Times  required  to  execute  a  preprogrammed  query  (M,  o). 

b.  Times  required  to  generate  new  queries  (M,  o). 

c.  Times  required  for  the  system  to  provide  a  response  to  a  query  (time  from 
execution  to  receipt  of  response,  M,  a). 

d.  Number  of  times  the  system  did  not  accept  analyst  generated  queries,  by 
operator,  by  shift. 

e.  Times  required  from  transmission  of  sensor  tasking  message  to  receipt  of 
response  at  analyst  terminal  (M,  o ).. 

f.  Number  of  errors  made  in  completing  formatted  messages  by  message  type 
(N,  %). 

g.  List  and  description  of  circumstances  surrounding  any  injuries  to  personnel. 

h.  List  of  any  hardware  or  software  damages  due  to  operator  error. 


